After graduating from high school, Kaufman received an athletic scholarship to Iowa State University. He then worked his way through the minors with ease, even setting the International League record for saves in a season (25). He was then recognized for his efforts by the Yankees, who called him up in 1982. He remained a Yankee through the 1983 season, but was traded to the California Angels after the season for infielder Tim Foli. He only played one season with the Angels; his career was cut short by a career ending elbow surgery. He retired after the surgery with a career ERA of 4.48, 50 strikeouts, a save, and an overall record of 3-3.