This was a period of serious unrest and civil war, and the castle was slighted in 1155, early in the reign of King Stephen's successor, King Henry II.
During the Civil War, the castle's walls and defences were in such a poor state of repair that the Parliamentarian army did not bother to slight them when they gained control of the castle from the Royalists.