He learned piano and tuba as a child and later picked up the bass guitar. He learned arranging while serving in the United States Army.
Following stints with Coleridge Davis and Stuff Smith in the 1940s, in 1951 Pate was recording on Chess Records with Eddie South and his Orchestra, credited on bass and arrangements. This was also the first of a series of Chess recordings on which Pate collaborated with saxophonist Eddie Johnson. In the 1950s he was also a resident arranger for Red Saunders's house band at the Club DeLisa.
Johnny Pate's trio recorded for a number of Chicago labels, including Gig and Talisman. For the Cincinnati-based Federal Records, the Johnny Pate Quintet had a hit with "Swinging Shepherd Blues", which reached No. 17 on the Billboard R&B chart in spring 1958.
One of the last albums on which Pate played bass was James Moody's 1958 album Last Train from Overbrook, on the Chess subsidiary, Argo Records.
Pate produced and did the arrangements for B.B. King's album Live at the Regal in November 1964. Pate was also the arranger and conductor for Wes Montgomery's album Movin' Wes, released in 1965 and re-released in 1981. He was the arranger and conductor for Lu Elliott's Sings Way Out From Down Under 1967 ABC album.
In the early 1960s, Pate was hired by OKeh Records producer/A&R director Carl Davis to write arrangements for the label. Davis had had previous hits with artists such as Walter Jackson, Major Lance, Ted Taylor and The Opals.
Pate, Curtis Mayfield and The Impressions first teamed in January 1963 recording the ballad "Sad Sad Girl and Boy," which mid-charted in Cashbox magazine's charts. The following single, "It's All Right," stayed at number one R&B for two weeks and hit number four pop in fall 1963; it was followed by "Talking about My Baby," "I'm So Proud," and "Keep on Pushing". The Keep on Pushing LP peaked at number eight pop in fall 1964. Pate produced and recorded most of their hits at Universal Recording Studios in Chicago.
Their success led the group's label, ABC-Paramount, to open a Chicago office on 14th and Michigan and appoint Pate as A&R director in 1964. One of the acts he signed, the Marvelows, had a number seven R&B hit with "I Do". "I Do" was followed by another hit with "In the Morning."
Other acts signed to ABC-Paramount through the Chicago branch were the Trends, the Kittens, and former Vee-Jay Records star Betty Everett. He also did the arrangements for Major Lance's Monkey Time.
In 1968, Pate began doing arrangements for Curtis Mayfield's Curtom label. Leaving in 1972, he worked on numerous recordings including the horn arrangements for the Bobby Bland and B.B. King Together Again...Live (1976), produced and arranged several albums for Peabo Bryson on Capitol Records, including Gold Award album CrossWinds in 1978, and the 1978 album Words and Music by Lonette McKee on Warner Bros. Records. Pate also scored soundtracks for films including Shaft in Africa (1973), Bucktown (1975), Satan's Triangle (1975), Dr. Black, Mr. Hyde (1976), Sudden Death (1977) and Every Girl Should Have One (1978).
Pate also did the arrangements for Bee Gees' 1973 album Life in a Tin Can.
In 2006, TNC Records released an 80th birthday tribute album. His song "Shaft in Africa", was sampled by producer Just Blaze, for the Jay-Z single "Show Me What You Got". It was later sampled by producer K-Def for Diddy's "We Gon' Make It", featuring Jack Knight.
In the late 1960s Pate served as a national trustee on the National Academy of Arts and Sciences and he was very instrumental in bringing The Grammy Awards to television.Johnny Pate Trio (1956)
Subtle Sounds (1956)
Johnnie Pate at the Blue Note (1957)
Jazz Goes Ivy League (1958)
Swingin' Flute (1958)
A Date With Johnnie Pate (1959)
Set A Pattern (1968)
Superfly Conductor (1972)
Shaft in Africa (1973)
With James MoodyLast Train from Overbrook (Argo, 1958)