According to the legend, the spirit of "Annie Palmer" haunts the grounds of Rose Hall Plantation near Montego Bay. The story states that she was born in Haiti to an English mother and Irish father and spent most of her life in Haiti. When her parents died of yellow fever, she was adopted by a nanny who taught her witchcraft and voodoo. She moved to Jamaica and married John Palmer, owner of Rose Hall Plantation. Annie murdered Palmer along with two subsequent husbands and numerous male plantation slaves, later being murdered herself by a slave named "Takoo". A song about the legend called "The Ballad of Annee Palmer" was recorded by Johnny Cash. For many years Cash owned the nearby Cinnamon Hill Great House.
Geoffrey S. Yates, Assistant Archivist, Jamaica Archives, claimed that the false story started with an account by Rev. Hope Masterton Waddell of the strangling of Mrs. Palmer at the adjacent Palmyra Estate in 1830. However, the passage in Waddell's memoirs simply includes a footnote claiming, "The estate furnished scenes and characters for Dr. Moore's novel Zeluco. The cellars and spikes used by a lady owner therefor the necks of her slaves I have seen, and also the bed on which she was found dead one morning, having been strangled".: 53 However, the novel has an anti-slavery theme, and the only scenes set in the Caribbean are located in Cuba and feature none of the details claimed by Waddell. Waddell, himself an abolitionist, was also writing in the context of the Baptist War of which Waddell was a first-hand witness. He stated that the Palmyra Estate was set on fire alongside the Kensington Estate, located further in land as a signal for a general insurrection.
The legend was elaborated by the journalist John Castello in 1868. Castello was the owner of the local Falmouth Post when he published a small pamphlet Legend of Rose Hall and erroneously describes a memorial in St. James Church to "Anne Palmer".
An investigation of the legend in 2007 by Benjamin Radford concluded that the story was fictionalized and modeled on the title character in a famous Jamaican novel, The White Witch of Rosehall by Herbert G. de Lisser, published in 1929. An Annie Palmer unrelated to Rose Hall did exist, and by all accounts had no tendencies toward sadism or lechery. Rough Guide To Jamaica author Polly Thomas writes that the name of Annie Palmer may have become confused with Rosa Palmer, the original mistress of Rose Hall who did have four husbands but was said to be unwaveringly virtuous. none of Mrs. Rose Palmer's husbands died by murder or under any suspicious circumstances. The real Mrs. Annie Palmer (nee Paterson), was the Jamaican-born wife of the Custos of St. James, and daughter of Dr. Paterson, Custos of Hanover, and his wife. Neither of her two husbands died of murder, and she was not murdered. The Patersons were of Scottish descent, and close relatives of the ancient Paterson of Eccles family of Scotland.
- Barry Reckord's 1975 play White Witch of Rose Hall is based on the legend.
- The television series Ghost Adventures featured an episode at Rose Hall.
- The 1993 novel Voyager by Diana Gabaldon features Rose Hall as a setting while the main characters are in Jamaica.
- Coven featured the song "The White Witch of Rose Hall" on their first album, Witchcraft Destroys Minds & Reaps Souls (1969).
- The 19th-season finale of America's Next Top Model featured its final runway show taking place at Rose Hall.
- Scariest Places on Earth featured Rose Hall in Season 2 Episode 3, "White Witch".
- Ghost Brothers featured Rose Hall in Season 2 Episode 5, "Rose Hall"
- Ghost Hunters International featured Rose Hall season 2 episode 13
- The Dead Files covered the legend in season 4, episode 1. Amy Allan, the show's lead investigator, claimed to encounter at least two female ghosts and identified them as Rosa and Annie Palmer.
- Outlander featured Rose Hall in Season 3, Episode 13
- cEvin Key of Skinny Puppy released a solo album entitled The Ghost Of Each Room, which features the famous painting on the cover
- Yates, Geoffrey S. "The Rose Hall Jamaica legend". www.jamaicanfamilysearch.com. Retrieved 4 June 2019.
- Johnny Cash; Don Cusic (25 August 2004). Johnny Cash: The Songs. Da Capo Press. pp. 226–. ISBN 978-1-56025-629-8. Retrieved 7 February 2013.
- "Jamaica Great Houses, a symbol of the Plantation Era". jamaicagreathouses.com. Retrieved 5 June 2019.
- Waddell, Hope Masterton (1863). Twenty-nine years in the West Indies and Central Africa: a review of missionery work and adventure. 1829-1858. Harvard University. London and New York: T. Nelson and Sons.
- Radford, Benjamin (September 2008), "The White Witch of Rose Hall", Fortean Times, no. 239, ISSN 0308-5899
- Jim Stefko, "The White Witch of Rose Hall", Suite101.com (site blocked)
- "Heeding (Or Ignoring) Skeptical Investigation". Center for Inquiry. 2011-10-22. Retrieved 2011-11-14.
- Polly Thomas (2 August 2010). The Rough Guide to Jamaica. Penguin. pp. 185–. ISBN 978-1-4053-8205-2. Retrieved 31 May 2012.
- Michael Davis, "White Witch, Chelsea Theatre – Review", Breaking The Fourth Wall, 10 April 2017.
- "Rose Hall : Ghost Adventures". Travel Channel. Retrieved 2013-02-07.