Born in St. Louis, Missouri of German and Irish descent, Pankow moved with his family to Park Ridge, Illinois at the age of eight, where he started playing the trombone at St. Paul of the Cross Elementary School.
One of nine siblings, Pankow was influenced by his musician father, Wayne. He is the older brother of veteran actor John Pankow. His Notre Dame High School band instructor was Father George Wiskirchen, CSC
Pankow earned a full music scholarship to Quincy College, where he studied the bass trombone. After completing his first year, he returned home for the summer and formed a band that began to play some live local shows. Not wanting to give up this work, Pankow transferred to DePaul University.
He is a member of Phi Mu Alpha Sinfonia, and was given the National Citation as well as recognition as Signature Sinfonian along with fellow Chicago members and Sinfonians on August 26, 2009.
At DePaul, Pankow met Walter Parazaider, who recruited him to join a band named The Big Thing, which would become Chicago Transit Authority. Soon after the first album's release, the band's name was shortened to Chicago. Pankow has remained a member of Chicago since its inception.
In addition to playing the trombone, Pankow has composed many songs for Chicago, including the hits "Make Me Smile" and "Colour My World" (both from his suite Ballet for a Girl in Buchannon), "Just You 'N' Me," "(I've Been) Searchin' So Long," "Old Days," "Alive Again," and (with Peter Cetera) "Feelin' Stronger Every Day."
Pankow has also composed most of Chicago's brass arrangements over the years.
Although Pankow is not one of the band's principal vocalists, he sang lead vocals for two Chicago songs: "You Are On My Mind" (from Chicago X, 1976) and "Till the End of Time" (Chicago XI, 1977).
Along with fellow Chicago horns Lee Loughnane and Walter Parazaider, Pankow was featured on Three Dog Night's 1969 #15 hit "Celebrate" and on several tracks of the 1979 Bee Gees' Spirits Having Flown album. Pankow has appeared on several albums for the rock band Toto, including the 1982 Grammy Award winning Toto IV and their 2006 album Falling in Between, for which he composed the brass arrangements and performed on the song "Dying on My Feet".
James Pankow and his wife Jeanne Pacelli have two children, Carter (born 1999) and Lilli (born 2002). His marriage to first wife Karen lasted 18 years and produced two children, Jonathan (born 1981) and Sarah (born 1986).