Highly unlikely I'm sure, but did James have any relatives working tollgates near Exeter ?
See bottom-lefthand "Loggerheads" on this link.
http://www.exetermemories.co.uk/em/_transport/tollhouses.php

Glad you've found my other suggestions of interest, all the best to you and yours.

(ps a little "spooky" this - I've just realised that immediately before logging into the forum today and seeing your post, I was emailing a friend to watch a "Who Do You Think You Are?" repeat on tv tonite. And yes that episode is about a Gibb, in this case Robin. Maybe the surname's not as unusual as I was thinking a mere 10 mins ago !)

J