If after 1851, one can perhaps trace with census data. Do you have any dates, ages, parishes, adresses or names of other family members.
For example in 1861 there are two manx born James Caleys, one in Everton the other in Liverpool.