Did you find Elizabeth in the later censuses, and when she died? He must have been home often enough to produce several children.
I thought Elizabeth's sister Jane Fleming, widow aged 28, nurse b.IOM, might give a lead as to where they were later. I also looked for her on Chart 14 on the Qualtrough website, but not there. According to the IGI a Jane Qualtrough married JAMES BOYD FLEMING on 30 SEP 1849 Braddan - so very young if she was the sister in 1861, and her age was correct - but I can't find her in England later, either.
If the NZ descendant found Charles Edwards and their marriage I'll be interested to see how the mystery was solved.