baronet | Baronet | Sir Arthur Harris, 1st Baronet | Sir Fitzroy Maclean, 1st Baronet | Sir Francis Baring, 1st Baronet | Sir William Temple, 1st Baronet | Sir Richard Fanshawe, 1st Baronet | Sir William Johnson, 1st Baronet | Sir Robert Eden, 1st Baronet, of Maryland | Sir Orlando Bridgeman, 1st Baronet, of Great Lever | Sir John D'Oyly, 1st Baronet, of Kandy | Sir James Fergusson, 6th Baronet | Sir Henry Rawlinson, 1st Baronet | Sir Frederick Pollock, 3rd Baronet | Sir Charles Trevelyan, 1st Baronet | Sir William Williams, 1st Baronet, of Kars | Sir Jonathan Trelawny, 3rd Baronet | Sir John Gell, 1st Baronet | Sir Henry Thompson, 1st Baronet | Sir Harry Burrard-Neale, 2nd Baronet | Sir Godfrey Copley, 2nd Baronet | Sir George Beaumont, 7th Baronet | Sir Frederick Pollock, 1st Baronet | Sir Francis Bernard, 1st Baronet | Sir Edmund Beckett, 4th Baronet | Sir William Mackinnon, 1st Baronet | Sir William Langhorne, 1st Baronet | Sir William Ingram, 1st Baronet | Sir Thomas Lyttelton, 4th Baronet | Sir Thomas Herbert, 1st Baronet |
His second wife was Lady Mary Arabella Lumley, daughter of Richard, 4th Earl of Scarborough and Barbara, younger daughter and co-heiress of Sir George Savile, last Baronet of Rufford.
In 1867 he and his father assumed by Royal license the additional surname of Fermor and in 1876 he succeeded his elder brother as 7th Baronet of Rufford.