He was born in Wechold, near Hoya, and his father, also called Philipp Spitta, was a theologian and wrote the Protestant collection of hymns entitled Psalter und Harfe. As a child, the younger Spitta learnt the piano, pipe organ, and musical composition. He studied theology and classical philology at the University of Göttingen from 1860, graduating in 1864 with a Ph.D. for a dissertation on Tacitus (Der Satzbau bei Tacitus, 1866). While at university, he composed, wrote a biography of Robert Schumann, and became friends with Johannes Brahms. He became a teacher of Ancient Greek and Latin language in, successively, Reval, Sondershausen, and Leipzig, while pursuing his interest in and lecturing on music history in general and Johann Sebastian Bach in particular.
His Bach study began to be published in 1873, and was followed by an appointment as professor of music history at the University of Berlin in 1875, and a further appointment as administrative director of the Berlin Hochschule für Musik, at which posts he remained for the rest of his life. The students he taught include Oskar Fleischer, Max Friedlaender, Carl Krebs, Max Seiffert, Agnes Tschetschulin, Emil Vogel, Peter Wagner, and Johannes Wolf. He founded one of the first scholarly music periodicals, the Vierteljahrsschrift für Musikwissenschaft, with Friedrich Chrysander and Guido Adler in 1885, and also had an important role in the publication of the Denkmäler deutscher Tonkunst.
He left a strong influence on the new fields of historical criticism and musicology; his work spanned periods of music history from the early Middle Ages to his own time, and embraced research, teaching, writing, and editing of musical editions to a very rigorous degree, including the use of source-critical studies. He was influenced by neo-Kantian philosophy. In his Bach biography, he wrote the first major study of German choral and keyboard music of the 17th century (early baroque).
Most of his papers are divided between the library of the Berlin Hochschule für Musik, the Staatsbibliothek zu Berlin, and the library of the University of Łódz. He contributed many scholarly articles to periodicals, and wrote articles on Schumann, Spontini, and Weber for Grove's Dictionary of Music and Musicians in 1886.
- Ein Lebensbild Robert Schumanns (Leipzig, 1862)
- Johann Sebastian Bach (Leipzig, 1873–1880, 1962; English translation: Johann Sebastian Bach. His Work and Influence on the Music of Germany, 1685 - 1750 (Clara Bell and J. A. Fuller-Maitland, transl.), 1884–1885, 1899)
- Zur Musik (Berlin, 1892) - 16 essays
- Musikgeschichtliche Aufsätze (Berlin, 1894) - collected essays
- U. Schilling: Philipp Spitta: Leben und Wirken im Spiegel seiner Briefwechsel (Kassel, 1994) - contains a complete bibliography of Spitta's writings
- H. Riemann: Philipp Spitta und seine Bach-Biographie (Berlin, 1900)
- Johannes Brahms: Briefwechsel XVI (Berlin, 1920) - contains Brahms-Spitta correspondence
- W. Sandberger: Das Bach-Bild Philipp Spittas: ein Beiträg zur Geschichte der Bach-Rezeption im 19. Jahrhundert (Stuttgart, 1997)
- Encyclopedia Americana. 1920. .