An original member of pioneering British jazz-rock band, If (1969–1973), he went on to undertake session and studio work. Around 1975–76 he was also a member of Pip Pyle's The Weightwatchers, with Elton Dean and Keith Tippett and leading his own group, Jim Richardson's Pogo Revisited which also featured Alan Barnes. In the late 1970s, his quartet featured Bobby Wellins. In 1981 he appeared on the Hoagy Carmichael tribute album In Hoagland.
In the mid-1980s he reunited with former If band member Dick Morrissey in Morrissey–Mullen. He worked with Dexter Gordon and Chet Baker and was a member of the Jack Honeyborne Quintet with Derek Wadsworth, Vic Ash, and Tony Kinsey.
Richardson released the album Chapter One on 15 October 2009. His album 2 Plus 2 was released on 22 April 2010. He was one of the guest speakers for the launch of the Princess Royal Trust for Carers Out of Hospital report held on 21 July 2010 at the London Medical Society. He had been a carer for his partner, Maggie, before she died of cancer. Richardson provided a carers perspective to the gathered professionals, on behalf of the PRTC, to help campaign for more consideration by hospitals, to the needs of carers during patient discharge planning.
- Don't Get Emotional (Spotlite, 1988)
With Chet Baker
- Rendez-Vous (Bingow, 1980)
- Live in London Vol. 2 (Ubuntu Music 2018)
- Live in London (Ubuntu Music 2016)
- If (Island, 1970)
- If 2 (Island, 1970)
- If 3 (United Artists, 1971)
- If 4 (United Artists, 1972)
- Waterfall (Metromedia, 1972)
- Europe '72 (Repertoire, 1997)
- Fibonacci's Number: More Live (Repertoire, 2010)
- Hoagy Carmichael, Georgie Fame, Annie Ross, In Hoagland 1981 (Bald Eagle 1981)
- Frank Evans, Mark Twain (77 Records 1970)
- Rachel Gould, Chet Baker, All Blues (Bingow, 1979)
- Barbara Jay, Lee Gibson, Tina May, The Ella Fitzgerald Songbook Revisted (Spotlite, 2000)
- Lloyd Ryan, Jazz 4 New Directions (Playback)
- Helen Shapiro, Straighten Up and Fly Right (Dureco 1983)
- Tommy Whittle, Grace Notes (Spotlite, 2003)
- Tommy Whittle, The Tenor Connection (Spotlite, 2010)