For many years it was owned by the Dean and Chapter of St Paul's Cathedral, who let it out to the Grey family.