Ruthardt was born in Stuttgart to Friedrich Ruthardt, who was an oboist and composer. The younger Ruthardt became a violinist in the Stuttgart court orchestra at a young age in 1855. Later, he worked in a number of cities as Kapellmeister: Riga from 1871 to 1882, Leipzig from 1882, Berlin from 1884, Bremen from 1893, and Berlin again from 1898. He retired to Konstanz in 1900, where he died in 1909. Among his compositions, the incidental music for Bjørnstjerne Bjørnson's Halte-Hulda is notable.