The earlier census records point to a c1850 birth including this one that includes wife Elizabeth:-

1871, Market Street, Douglas, IOM, James Stewart, Head, married, 19, labourer, Ramsey IOM
- Wife, Elizabeth, 17, house keeper, Liverpool

Here's a transcript of the 1850 baptism:-

Baptisms solemnized in the Parish of Maughold Chapelry of Ramsey in the Isle of Man in the year 1850

Priv Bap
No. 121.
When Baptized: 8 October 1850
Child’s Christian Name: James
Parents Name: Samuel & Margaret Stewart
Abode: Maughold Street Ramsey
Quality, Trade or Profession: Labourer
By whom the ceremony was performed: William Kermode, chaplain

Link: familysearch.org/ark:/61903/1:1:XZYX-XMV