'The only thing I still don't know is the whereabouts of Lydley'

The place name Lydley appears to be derived from the 1891 census return. The 1911 census return clearly states the birthplace as Lydney. I would suggest that the 1891 census enumerator has made an error and that the place name should have been Lydney on the 1891 return.