This part of Wales, including the villages of Clyro, Capel-y-ffin, Llowes, Glasbury, Llanigon, Painscastle, and the town of Hay-on-Wye, as well as Clifford and Whitney-on-Wye in neighbouring Herefordshire, is sometimes referred to as "Kilvert Country".
Williams was born in 1856 in Llowes, Radnorshire, the second child of the Reverend Garnons Williams of Abercamlais, Powys, and his wife Catherine Frances, the daughter of Fenton Hort, of Leopardstown, Dublin, and sister of Fenton John Anthony Hort.