Between 1856 and 1866, John Oliver built railways in Italy, with the unification of Italy and the revolutions that accomplished it in the background. During this time, he also did engineering jobs in England and in the early1860's became involved in mining in Cornwall in partnership with his son Henry Kinnaird, He also built a short railway line there.
During John Oliver's time in France, the girls would have been there in school. The boys were likely in school in England. When he moved to Rome, it is possible the two older girls at least came with them and Oliver was working with his father. Helen Maria was either there as well or visited as, in later years in Canada, she spoke of being in Italy. Sometime before 1860, John Oliver purchased a house in London near Hyde Park. Helen Maria and Henry Kinnaird were at this residence in 1861 with one female servant but, as their parents and other siblings were not, and didn't appear in any other England census records, they were all presumably on the continent.
In 1862, Frances Elizabeth married Paolo Ruspoli, a member of the Ruspoli Princes of Cervetes, a very old and noble Roman family. Paolo was a Barrister in Paris at the time but they subsequently moved to Italy where three daughters were born. (One of them, Francesca, corresponded with Helen Maria). Frances Elizabeth died in 1874 - likely in childbirth.