At the age of 17, he joined the San Francisco Symphony as a cellist. He subsequently played cello with the Los Angeles Philharmonic, and the Chicago Symphony Orchestra before becoming principal cello of the New York Philharmonic under Arturo Toscanini in 1929. He frequently performed with these orchestras as a soloist.
Toscanini, also a cellist, advised Wallenstein to become a conductor. He conducted the Hollywood Bowl Orchestra and then conducted frequently on the radio. From 1943 to 1956, he was music director of the Los Angeles Philharmonic. He later taught at the Juilliard School in New York, where he died in 1983 at age 84.
As solo cellist he recorded Strauss's Don Quixote under Beecham in 1932. For Audio Fidelity in September and October 1958, he conducted the specially-formed Virtuoso Symphony of London orchestra in Walthamstow Town Hall.
The LPs included Brahms's 4th Symphony, Tchaikovsky's Pathetique, Berlioz's Symphonie Fantastique, Pictures at an Exhibition, Boléro, Romeo and Juliet, the Nutcracker Suite and a suite from Carmen.
He was a descendant of Albrecht von Wallenstein.