Wexler was born in The Bronx, New York City, had a Jewish father of Polish ancestry and mother of German origin, and grew up in the Washington Heights neighborhood of Upper Manhattan. Despite graduating from George Washington High School at age 15, he dropped out of the City College of New York after two semesters. In 1935, Wexler enrolled at what is now Kansas State University and dropped out several times. Following his service in the Army, Wexler became a serious student, and he graduated from Kansas State with a B.A. in journalism in 1946.
During his time as an editor, reporter, and writer for Billboard Magazine, Wexler coined the term "rhythm and blues". He became a partner in Atlantic Records in 1953. There followed classic recordings with Ray Charles, the Drifters and Ruth Brown. With Ahmet and Nesuhi Ertegün, he built Atlantic Records into a major force in the recording industry. In 1967 he was named Record Executive of the Year for turning Aretha Franklin's career around.
In the 1960s, he notably recorded Wilson Pickett and Aretha Franklin, and oversaw production of Dusty Springfield's highly acclaimed Dusty in Memphis and Lulu's New Routes albums. He also cultivated a tight relationship with Stax Records, was an enormous proponent of the then-developing Muscle Shoals Sound and founded the fortunes of Muscle Shoals Sound Studios and the Muscle Shoals Rhythm Section. His work in this decade put Atlantic at the forefront of soul music.
In 1968, he and Ahmet Ertegun signed Led Zeppelin to Atlantic Records on the recommendation of singer Dusty Springfield and from what they knew of the band's guitarist, Jimmy Page, from his performances with The Yardbirds.
In 1975, Wexler left Atlantic Records for Warner Bros. Records.
In 1979, Wexler produced Bob Dylan's controversial first "born again" album, Slow Train Coming at Muscle Shoals; a single from that album, "Gotta Serve Somebody", won a Grammy award in 1980. When Wexler agreed to produce, he was unaware of the nature of the material that awaited him. "Naturally, I wanted to do the album in Muscle Shoals - as Bob did - but we decided to prep it in L.A., where Bob lived", recalled Wexler. "That's when I learned what the songs were about: born-again Christians in the old corral... I like the irony of Bob coming to me, the Wandering Jew, to get the Jesus feel... [But] I had no idea he was on this born-again Christian trip until he started to evangelize me. I said, 'Bob, you're dealing with a sixty-two-year-old confirmed Jewish atheist. I'm hopeless. Let's just make an album.'"
In 1983, Wexler recorded with UK pop star George Michael. The most famous outtake of these sessions would prove to be a rare early version of "Careless Whisper", recorded in Muscle Shoals.
In 1987, Wexler was inducted into the Rock and Roll Hall of Fame. He retired from the music business in the late 1990s.
Wexler spent a majority of the 1990s living on David's Lane in East Hampton, NY, where he shared living spaces with a Chinese family who aided him with daily functions and kept him company.
In Ray, the biopic of Ray Charles, Jerry Wexler is portrayed by Richard Schiff.
Interviews and archive footage of Wexler are featured prominently in the 2000 documentary film Immaculate Funk, which explores the roots of the classic American R&B and soul music.
Jerry Wexler died at his home in Sarasota, Florida, on August 15, 2008, from congestive heart failure. Asked by a documentary filmmaker several years before his death what he wanted on his tombstone, Wexler replied "Two words: 'More bass'."