I have found an electoral record that suggests he was still in Primrose Hill, Lydney in 1926

I assume that's a typo as he died in 1922

There is a record of him living in Primrose Hill in 1922. However, his death was registered in Llanelly so he may not have died in Lydney and might not be buried there. I would get his death certificate which should clarify things.