He married Catherine Southwell in 1790 and fathered two children, George and Sophia (later, the Baroness de Clifford).
On his retreat from Scotland, Balliol sought refuge with the Clifford family, land owners in Westmorland, and stayed in their castles at Appleby, Brougham, Brough, and Pendragon.
In 1729 his widow and her four sisters became co-heiresses to the Barony of Clifford.
Her grandson inherited the much older and more distinguished Barony of Clifford as its 20th holder.
There are monuments in the church dating from the 16th and 17th centuries to the memory of members of the Clifford family.
Baron | William Cecil, 1st Baron Burghley | baron | Sacha Baron Cohen | Joseph Lister, 1st Baron Lister | Joey Baron | Robert Baden-Powell, 1st Baron Baden-Powell | Henry Brougham, 1st Baron Brougham and Vaux | Guy Carleton, 1st Baron Dorchester | Clifford Evans | Clifford Odets | William Wallace, Baron Wallace of Saltaire | William Tyssen-Amherst, 1st Baron Amherst of Hackney | James Mackay, Baron Mackay of Clashfern | Hercules Robinson, 1st Baron Rosmead | George Brydges Rodney, 1st Baron Rodney | Beechcraft Baron | Thomas Seymour, 1st Baron Seymour of Sudeley | Thomas Denman, 1st Baron Denman | Stuart Rendel, 1st Baron Rendel | Norman Foster, Baron Foster of Thames Bank | John Moore-Brabazon, 1st Baron Brabazon of Tara | Charles Cochrane-Baillie, 2nd Baron Lamington | Charles Calvert, 5th Baron Baltimore | Benedict Calvert, 4th Baron Baltimore | Baron Hans Heinrich Thyssen-Bornemisza | William Ponsonby, 1st Baron Ponsonby | Walter Rothschild, 2nd Baron Rothschild | Sidney Herbert, 1st Baron Herbert of Lea | Robert Napier, 1st Baron Napier of Magdala |