Haley was born in Boston, Massachusetts, to Canadian-born parents John Joseph Haley Sr. and Ellen Curley Haley. His father was a sailor by trade and died in a ship wreck off the coast of Nova Scotia on February 1, 1898, when Jack was only six months old. He had one older brother, Bill, who died of pneumonia in 1915 at the age of 20 after contracting tuberculosis.
Haley headlined in vaudeville as a song-and-dance comedian. One of his closest friends was Fred Allen, who would frequently mention "Mr. Jacob Haley of Newton Highlands, Massachusetts" on the air. In the early 1930s, Haley starred in comedy shorts for Vitaphone in Brooklyn, New York. His wide-eyed, good-natured expression gained him supporting roles in musical feature films, including Poor Little Rich Girl with Shirley Temple, Higher and Higher with Frank Sinatra and the Irving Berlin musical Alexander's Ragtime Band. Both Poor Little Rich Girl and Alexander's Ragtime Band were released by Twentieth Century-Fox. Haley was under contract to them and appeared in the Fox films Rebecca of Sunnybrook Farm and Pigskin Parade, marking his first appearance with Judy Garland.
Haley hosted a radio show from 1937 to 1939 known to many as The Jack Haley Show. The first season (1937-1938), the show was sposored by Log Cabin Syrup and was known as The Log Cabin Jamboree. The next season (1938-1939), the show was sponsored by Wonder Bread and was known as The Wonder Show. During the second season the show featured Gale Gordon and Lucille Ball as regular radio performers.
Haley returned to musical comedies in the 1940s. Most of his '40s work was for RKO Radio Pictures. He left the studio in 1947 when he refused to appear in a remake of RKO's Seven Keys to Baldpate. Phillip Terry took the role. He subsequently went into real estate, taking guest roles in television series over the next couple of decades.
Metro-Goldwyn-Mayer hired Haley for the part of the Tin Man in The Wizard of Oz after its contracted song-and-dance comedian Buddy Ebsen suffered an almost fatal allergic reaction. He had unwittingly inhaled some of the aluminum dust that composed the majority of the components that went into the creation of his silver face makeup. Bits of it instantly began to settle on his lungs and within a few days of principal photographic testing, he found himself preparing to sit down to dinner one night only to encounter difficulties taking a regular breath of oxygen. The dust was subsequently converted into a paste for Haley in the hope that the previous catastrophe that befell Ebsen would not be repeated. This time around, however, a different incident occurred. The application of the aluminum paste to Haley's face resulted in an eye infection that led to his being off the set for four days of shooting. Appropriate surgical treatment was administered and any chance of serious or permanent eye damage averted. Haley also portrayed the Tin Man's Kansas counterpart, Hickory, one of Aunt Em and Uncle Henry's farmhands.
Haley did not remember the makeup or the costume fondly. Interviewed about the film years later by Tom Snyder, he related that many fans assumed making the film was a fun experience. Haley said, "Like hell it was. It was work!" For his role as the Tin Woodman, Haley spoke in the same soft tone he used when reading bedtime stories to his children. Oz was one of only two films Haley made for MGM. The other was Pick a Star, a 1937 Hal Roach production distributed by Metro-Goldwyn-Mayer.
Haley was raised Roman Catholic. He was a member of the Good Shepherd Parish and the Catholic Motion Picture Guild in Beverly Hills, California. He married Florence McFadden (1902-1996), a native of Wilkes Barre, Pennsylvania on February 25, 1921, and they were married until his death. Flo Haley opened a successful beauty shop and had many film personalities among her clients. The couple had a son, Jack Haley, Jr. (1933-2001), who became a successful film producer, and a daughter, Gloria (1923–2010). In 1974, the younger Haley married entertainer Liza Minnelli, the daughter of his father's Oz co-star Judy Garland. The marriage ended in divorce in 1979. Jack Haley, Jr. died on April 21, 2001. Gloria Haley-Parnassus died on May 1, 2010. His nephew Bob Dornan served as a Republican congressman from California.
Haley's last film appearance was in 1977's New York, New York—in the lavish "Happy Endings" musical number, he played a host who introduces a top Broadway star at an award ceremony, played by his then-daughter-in-law, Liza Minnelli. Two years later, Haley died of a heart attack on June 6, 1979, in Los Angeles, California. Two months prior, on April 9, 1979, he appeared at the 51st Academy Awards ceremony with his Oz co-star Ray Bolger to present the award for Best Costume Design. Bolger announced the nominees, Haley the winner. Before he could open the envelope, Bolger asked, "How come you get to read the winner?" Haley replied, "When your son produces the show, you can announce the winner". Jack Jr. was the show's producer that year. Haley remained active until a week before his death. He is buried in Holy Cross Cemetery, Culver City, California.
Haley's autobiography, Heart of the Tin Man, was published in 2000.