If the parish records states the parish/chapel is Westbury on Severn then it will be Westbury on Severn Parish Church i.e.St Mary, St Peter and St Paul, Westbury on Severn. This link may be helpful: http://www.achurchnearyou.com/westbury-on-severn-st-peter-st-paul/
If you use the link and go to the Find Us page you will see a link which will take you to a map displaying the parish boundaries as they are today.

Best wishes
Mike