have you got the marriage cert for James and Ann ? that should name his father - also where did they marry?
If on Island then have you looked at www.iMuseum.im - also Manx certs need be bought from IoM registry - see www.gov.im