Budden was born in Rockley, New South Wales, the son of Sarah Hale (née Stanger) and Arthur Budden.
The hamlet on the banks of the River Og is dominated by commuter housing and race horse stables on the narrow road that leads over the downs to Rockley.
The Ridgeway Benefice consists of the Wiltshire parishes of Holy Cross Chiseldon with Draycot Foliat, Ogbourne St Andrew, which also serves the hamlets of Ogbourne Maizey and Rockley, Ogbourne St George.