Ancestry has a "Thomas Scaett" age 21 Agl Labr born Douglas in the 1851 census at Bibaloe Moar Douglas. The head is William Graham age 71 Farmer of 169 acres. Looking at the census record, it's likely "Scarff".

I researched Scarff's years ago and my notes say I couldn't find him after 1841 either, with the possible exception of the above.