Gwinnett was born in 1735 in the parish of Down Hatherley in the county of Gloucestershire, Great Britain, to Welsh parents, the Reverend Samuel and Anne (née Button) Gwinnett.
Jemmy Wood, The Gloucester Miser, was a former owner of Hatherley House and estate.