In 1068 a Norman motte and bailey castle was built for William the Conqueror. During the Anarchy the castle was held by David, King of Scotland through his marriage to Maud, 2nd Countess of Huntingdon. He supported Empress Matilda in a battle of succession and the castle was much damaged at this time. However, when the succession was settled, David's son Henry paid homage to King Stephen who in turn gave him the borough of Huntingdon in addition to the castle.
By 1173 the castle was still held by the kings of Scotland, at that time by William the Lion. He had sided with Henry, the rebel son of Henry II, during the Revolt of 1173–1174, and the castle was besieged by Richard de Luci. The siege was taken over by Simon de St. Lis but it did not end until Henry II (who had just performed penance at the tomb of Thomas Becket) arrived at Huntingdon. The siege ended the following day and Henry then ordered that the castle be dismantled.
Parts of the castle did remain, including the chapel, and its ownership passed through a number of hands. The castle was refortified during the Civil War. For a time it served as the county gaol, and later a windmill stood on the castle mound. Nothing now remains of the castle itself apart from its earthworks, although the site is a scheduled ancient monument.