The earliest structure on the site was built for King William the Lion in the late twelfth century to repel Viking invasions to Lunan Bay. Evidence shows, however, that William took up residence there on several occasions whilst on hunting expeditions. In 1194, William conferred the castle, and land surrounding the village of Inverkeilor, 2 kilometres (1.2 mi) east of the castle, to Walter de Berkeley, the Great Chamberlain. On his death, his lands of Inverkeilor, with the castle, passed to Ingram de Balliol who had married the heiress of Walter. He rebuilt the castle and the property remained in that family for two generations. When his grandson, Ingram, who flourished between 1280 and 1284, died childless about 1305 the property passed to the son of Constance de Baliol, Henry de Fishburn.
The property was forfeit during the reallocation by Robert the Bruce who in 1328 gave the castle to the Earl of Ross. The castle is referred to as rubeum castrum (Latin for Red Castle) in deeds of 1286, referring to its burnished red sandstone, typical of this area.
In 1579, James, son of Patrick Gray, 4th Lord Gray, married Lady Elizabeth Beaton, who owned the castle, and fell in love with her daughter. After Lady Beaton threw him out, Gray (with his brother Andrew of Dunninald and John Ogilvy of Kynnody) laid siege to the castle for two years, ultimately burning the inhabitants out. James VI ordered John Erskine of Dun and his son Robert to bring siege engines and eject Gray, with the help of the townspeople of Dundee. On 28 February, a royal messenger was sent from Stirling Castle to make a proclamation at the mercat crosses of Dundee, Forfar, Montrose, and Arbroath ordering:
all our sovereign lord's lieges between sixty and sixteen years [old] to pass forward in warlike manner to the relief of the siege of the "hous of the Reidcastell", and to charge James Gray and John Ogilvie of Kynnody to leave the siege of the said house under the pain of treason.
John Erskine was asked to make an inventory of the goods in the castle and give safe conduct for Elizabeth Beaton's son, the poet John Stewart of Baldynneis to the king's presence. In March 1582 Andrew Gray was forgiven for setting fire to the castle and its yards, and other crimes.
In 1590 it was reported that 12,000 gold crowns had been landed in the creek near the castle to aid the Catholic cause in Scotland. The castle slipped into decline, and, although it remained partially roofed until 1770, it was never again a residence of nobility. Its last inhabitant was the minister of Inverkeilor, one James Rait.
Red Castle stands on high ground overlooking Lunan Bay, on the North Sea coast. Immediately to the north of Red Castle is the mouth of the Lunan Water, with the hamlet of Lunan beyond. Only a part of the fifteenth century rectangular tower, and the 2-metre (6.6 ft) thick east curtain wall remain. The tower in particular is in precipitous condition, being perched on the edge of the hill above Lunan Bay, and was described as being "in imminent danger of collapse" in 1999.
The castle is clearly visible from the A92 road and the Edinburgh to Aberdeen line. The remains are those of the 15th-century keep, and the surrounding wall, or enceinte, which may date from the 13th century. A midden below the castle is continually eroding, yielding a number of artefacts now in the Montrose Museum. The castle is protected as scheduled monument.
- Duchess of Cleveland, The Battle Abbey Roll with some account of the Norman Lineages, v.1, p. 253.
- Charles Thorpe McInnes, Accounts of the Treasurer of Scotland, vol. 13 (Edinburgh, 1978), p. 309.
- HMC 5th Report: Erskine (London, 1876), pp. 636, 640.
- Gordon Donaldson, Register of the Privy Seal: 1581-84, vol. 8 (Edinburgh, 1982), p. 128 no. 776.
- Calendar State Papers Scotland, vol. 10 (Edinburgh, 1936), p. 430.
- Historic Environment Scotland. "Red Castle, Site Number NO65SE 10.00 (35792)". Canmore. Retrieved 14 September 2009.
- Historic Environment Scotland. "Red Castle,Lunan Bay (SM2925)". Retrieved 21 December 2018.