Holden earned a Bachelor of Arts degree in English from Yale University in 1963. He worked as a photo editor, staff writer, and eventually became an A&R executive for RCA Records before turning to writing pop music reviews and related articles for Rolling Stone magazine, Blender, The Village Voice, The Atlantic, and Vanity Fair, among other publications. He first achieved prominence with his 1970s Rolling Stone work, where he tended to cover singer-songwriter and traditional pop artists. He joined the staff of The New York Times in 1981, and subsequently became one of the newspaper's leading theatre and film critics.
Holden's experiences as a journalist and executive with RCA led him to write the satirical novel Triple Platinum, which was published by Dell Books in 1980. He is the recipient of the 1986 Grammy Award for Best Album Notes for The Voice: The Columbia Years, a Frank Sinatra anthology. His poetry has been featured in The New Yorker and is included in the anthology The New Yorker Book of Poems.