A descendant of the Swabian House of Zähringen, he was the son of Margrave Herman V and Irmengard, daughter of Count Palatine Henry V of the Rhine. He succeeded his father in Baden on 16 January 1243.
In 1248, he married Gertrude of Austria, the niece of the last male member of the Babenberg dynasty, Duke Frederick II of Austria (1230–1246), and on the basis of that marriage claimed the duchies of Austria and Styria, leaving the rule over Baden to his younger brother Rudolf. However, he had a mighty rival in King Ottokar II of Bohemia, who in 1252 married Frederick's sister Margaret to legitimize his claims. According to the Privilegium Minus issued by Emperor Frederick Barbarossa in 1156, the Austrian lands could be bequeathed in the female line, and Herman even obtained the explicit consent by Pope Innocent IV. Nevertheless, the margrave and his son Frederick could not establish themselves in Austria and Styria against the resistance of the local nobility, who preferred Ottokar. Both rivals finally did not prevail, as the duchies were seized as reverted fiefs by the Habsburg king Rudolph I of Germany after Ottokar's death at the Battle on the Marchfeld in 1278.
The Margrave and his wife had the following children: