Silvers began his career in vaudeville in the early 1920s as a comedy partner of Phil Baker. As part of their act, Silvers would heckle Baker from the audience. The Baker/Silvers act was later used as the basis for the 1951 Martin and Lewis film The Stooge. The duo continued to perform together up through 1928.
In 1925 Silvers made his Broadway debut in the review Artists and Models. He also appeared in the review A Night in Spain in 1927 and contributed lyrics to the musicals The Song Writer (1928) and Pleasure Bound (1929). He wrote the book for the 1931 musical You Said It. He returned to the Broadway stage in 1932 to portray Louie Webb in the musical Take a Chance. He later wrote the music and lyrics to the review New Faces of 1936.
Silvers made his film debut in the 1929 feature The Show of Shows and then went on to play supporting roles in such films as Dancing Sweeties (1930), Bottoms Up (1934), Transatlantic Merry-Go-Round (1934), Born to Dance (1936), and Broadway Melody of 1936, notably also serving as a scriptwriter on the latter two films. He often contributed special comedy material to some of the larger MGM productions, including The Wizard of Oz in 1939.
In the 1940s Silvers was mainly active as a performer on the stage and on radio. He made one final film appearance in 1946, playing a featured comic role in Mr. Ace. In the 1960s he was a writer for The Mickey Rooney Show.
- Ruth Benjamin; Arthur Rosenblatt (2006). Who Sang What on Broadway, 1866–1996: The Singers (L-Z). McFarland & Company. p. 695.