After Napoleon's final defeat in 1815, Lord Keith retired from the Navy and the family lived mainly on their large estate at Tulliallan, where they had a magnificent castle constructed which is now the Scottish Police College.
The modern-day Tulliallan Castle is about a half mile to the southeast of the old castle, built in 1818–1820 for Viscount Keith.