Petrillo was born in Chicago, Illinois. Though, in his youth, Petrillo played the trumpet, he finally made a career out of organizing musicians into the union starting in 1919.
Petrillo became president of the Chicago Local 10 of the musician's union in 1922, and was president of the American Federation of Musicians from 1940 to 1958. He was ousted from his role as president when segregation became popular. It was becoming a more popular idea that the Local 10 (white musicians union) and Local 208 (black musicians union) would merge. He opposed this, which contributed to his dethroning.
The round-faced, bespectacled Petrillo dominated the union with absolute authority. His most famous actions were banning all commercial recordings by union members from 1942–1944 and again in 1948 to pressure record companies to give better royalty deals to musicians; these were called the Petrillo Bans.
Petrillo joined the orchestra at WBBM in Chicago, Illinois, in 1937. Before that, he had played trombone in the orchestras of Paul Whiteman and Vincent Lopez, among others. For a decade, he had been assistant conductor and orchestra member at three Chicago theaters. In 1940, he became the conductor of the WBBM orchestra. In 1943, he was promoted to music director at WBBM, "supervising all live and recorded music on the station."
Petrillo died October 23, 1984, at St. Joseph Hospital in Chicago. He was 92.
Petrillo was well known to the US and Canadian general public, especially during and after the 1942-44 recording ban and was frequently referenced in pop culture of the era. For example, Phil Harris, the band leader on the Jack Benny radio show, claims on the show to have been married to his wife, Alice Faye, by Petrillo. When Jack Benny asks how Petrillo could do this, Harris replies "Why not? My dues was paid up!" On another occasion Rochester is asked to blow the car horn by putting it in his mouth, and he replies "Petrillo won't let me!"
In the 1945 Crosby/Bergman film The Bells of St. Mary's, when Crosby's character, Father O'Malley, is asked how he was successful in tracking down a long-missing musician, he points to the sky and quips, "I went straight to the top—Petrillo!"
In 1945 or 1946, Robertson Davies had his "alter ego" write in a newspaper column, later collected in The Diary of Samuel Marchbanks, "Then to a party, where I showed my prowess at those games where you have to fill out forms saying who Cain's wife was, and whether it was Lincoln or Petrillo who said 'We must save the Union at all costs.'"
In the 1947 recording of "Huggin' And A Chalkin'" Johnny Mercer sings "...when I met another fella with some chalk in his hand coming around the other side (it was Patrillo)".
In the 1947 Bob Hope film My Favorite Brunette, a man says an imaginary George Washington is playing the fife all night long, and Hope replies "Why don't you tell Petrillo about it."
In the 1950 Warner Bros. animated short Hurdy-Gurdy Hare starring Bugs Bunny, the cartoon ends with Bugs making large amounts of money by having a (presumably non-union) monkey turn a street organ and an ape collecting the money from the apartment dwellers, during which he quips, "I sure hope Petrillo doesn't hear about this!" (The 1948 strike was ongoing at the time Hurdy-Gurdy Hare was in production.)
In the 1950 burlesque revue Everybody's Girl the comedians Bobby Faye and Leon DeVoe, playing anti-nudist street preachers, mention that the Devil has "two horns." DeVoe then jokes, "Two horns? Brother, we'll have to speak to Petrillo about that!"
Slim Gaillard's 1951 song "Federation Blues" (which is entirely about the musician's union) begins: "You may play an instrument and think that you're a killer / But you still ain't get nowhere till you see J.C. Petrilla."
In the 1952 Hope/Crosby film Road to Bali, Hope shows Crosby an instrument he's been using in his snake-charmer act. He quips, "Hey, I've been playing this flute all night. Have to clear it with Petrillo." There is a similar reference in the Hope picture My Favorite Brunette.
In the musical Sugar, two musicians witness the St. Valentine's Day Massacre. The gangsters are issued instructions to search for them in the song "Tear the Town Apart", which ends with "I'll call Petrillo".
The Petrillo Bandshell, in Chicago's Grant Park, is named after James Petrillo.
In the Lord Buckley riff "The Hip Gahn", there is a reference to "the Indian Petrillo" when the Indian people are planning a big musical party for Gandhi after independence is achieved.