Kompalla began his ice hockey career as a player with Gwardia Katowice in 1951. He won three Polish junior championships with Gwardia Katowice before playing for Gornik Katowice and Prussen Krefeld. He retired from playing in 1961.
He joined the International Ice Hockey Federation as an official in 1970. In 1972, Kompalla was selected to officiate the 1972 World Ice Hockey Championships matches between Czechoslovakia and the Soviet Union. That same year, he was also selected to officiate the Summit Series between the Soviet Union and Canada. His officiating partner during this series was Franz Baader. Both Kompalla and Baader received criticism from Team Canada during the series, which resulted in Kompalla pairing up with Rudolph Batja for the final game. During the final game, he called a penalty against Canadian J. P. Parisé which resulted in the player angrily swinging his stick in a threatening gesture at Kompalla. This resulted in a game misconduct for Parise, although Team Canada won the series.
He also officiated in several IIHF World Championships, Canada Cup competitions, and Winter Olympic Games. He was inducted into the IIHF Hall of Fame in 2003. That same year, Kompalla resigned as the referee representative in the German Ice Hockey League but stayed with the organization as an official match watcher.
- "Our heroes deserve life long recognition" (PDF). IIHF.com. November 2002. p. 6–7. Retrieved 1 March 2019.
- "Josef Kompalla". legendsofhockey.com. Retrieved 1 March 2019.
- Mulvoy, Mark (9 October 1972). "WAKING UP FROM A NIGHTMARE". si.com. Retrieved 28 February 2019.
- "Questions of Officials Brings Out New Soviet-Canada Ice Dispute". Fort Walton Beach Playground Daily News. 28 September 1972. Retrieved 28 February 2019.
- "Josef Kompalla tritt als Schiedsrichterbeauftragter zurück". eishockey.info (in German). 6 June 2003. Retrieved 1 March 2019.