I can find the family in 1871. He (Horatio N) is an innkeeper in Ramsey aged 36. She is 32 and they have 4 children born in Ramsey.
Howard 9, Charles E 6, Annie F 7 and Lilly 5.

I did not have a match for either Horatio or Ellen in the 1861 IOM census. There are a couple of possibilities for Horatio in England.
What I do not understand is why a spinster called Williams, would not have her father's family name (Brown).
Bruce