Two years after their arrival in Tasmania, when Henry was away from home, nine bushrangers came to their house. Mary was sitting with a baby on her knee when the men entered through the door and windows so that there was no escape. Threatening with a pistol they demanded jewellery, men's clothing and drink, of which there was none visible (the keg of home-made beer was in the cellar with the trapdoor covered by a carpet). After ransacking the house they left with their booty, and just when Mary had the kettle boiling one returned to take it from her, saying, 'It's full of water, just what I want'.¹⁶

After this incident Henry taught Mary to shoot and she became quite expert at it. As a future precaution she buried the remaining jewellery under a tree near the house, but could never find it again. Eventually the men were caught and eight were hung. The ninth was saved when it was proved that he refused to shoot someone. The family moved to a two-storey house at Battery Point about twelve months later, but they would frequently return to the farm in one of their two yachts, the Richmond packet and the Cockle shell. In Hobart the children went to a day school with the children of Henry's sister Charlotte Selina and her husband Alexander Bishop Butler. One day Mary went to the door for a delivery of bread and there encountered the ninth bandit, now doing honest work.¹⁷

From his time in Hobart, Henry appears on the Tasmanian Colonial Index as an employer of convicts between the years 1825 and 1835; as a witness at a trial in Richmond in 1833; and on the Jury lists from 1835 to 1840.¹⁸ His land grants from 1831 to 1833 were 1000 acres at York and 500 acres at Ralph's Bay Parish.¹⁹

In 1839 Henry organised for his family and that of his sister, Charlotte, to leave Tasmania on the brig Caroline, which was also carrying timber for their homes in Melbourne. On the way the ship was driven ashore on Swan Reef where it was unloaded, salvaged, and allowed to resume the voyage without the timber, arriving in Port Phillip on Monday 16 December 1839.²⁰ Henry had intended to leave his wife and children with his nephew John Blanch and his wife Sara. However, the next day two young men, Henry Griffin and Charles Deering, newly arrived in town, wandered into John Blanch's gunmaker's shop to purchase caps for their guns. One of them imprudently fired his gun forgetting that it was loaded. The result was an explosion which demolished the two-storey building and killed the young couple, John and Sara. Henry Griffin also died, but Charles Deering survived. Henry took the three Blanch children, who had been out walking with their nurse, to stay with his family.²¹

Henry and the boys had been in town for a week or so and had pitched their first tents on the south side of the river just below what is now Queens Bridge. During a flood their boxes were carried out into the bay, causing them to move to the other side of the river to where the Customs House now stands. Although Henry could not get a room for his own family he managed to find a new stable for his sister and her children.²² After a month a room was rented for Mary and baby Edmund until a small place near King Street was purchased. When more timber arrived a cottage was commenced in Flinders Lane between Queen and Market streets and the family was thankful to move in. The Blanch children stayed until March 1841 when Henry advertised for someone who was leaving for England to take charge of 'two stout and healthy boys of the ages of 5 and 7'.²³ A handsome reward was offered by Henry to anyone who was willing to escort the lads; he was sending his orphaned great-nephews, John and William, back to England to be with their grandparents.²⁴

Later Henry built a two-storey house at the corner of Collins and William streets, where the family stayed for several years. It was here that his son Matthew Henry died of fever in 1841.

Henry had a barque Favourite built on the Tamar River in Launceston, intending to take the family to England. He sold his property and they began the journey, firstly to see Sydney. Loath to let the ship sit idle whilst the family were enjoying themselves on shore, however, he sent her to Newcastle for a load of coal, but she stuck on a bar and did not return for six weeks. By this time Henry had decided that it would be too cold in England for him, and the family returned to Melbourne and 'Cranley Cottage' in Brighton. Henry later took the boys on a trip to California and when they returned the whole family went to Tahiti where they entertained Pomare IV, Queen of Tahiti, onboard the vessel. On their return from Tahiti, Henry gave the vessel to his son William (Mark William) who sailed first to Timor to buy ponies, and then on to Mauritius to sell the ponies, load up with sugar and return to Melbourne or Sydney.