Jackson's father, Al Jackson Sr., led a jazz/swing dance band in Memphis, Tennessee. The young Jackson started drumming at an early age and began playing on stage with his father's band in 1940, at the age of five. He later played in producer and trumpeter Willie Mitchell's band and at the same time was holding down a chair in the popular Ben Branch Band.
In an interview with Drum! magazine, Mitchell recalled,
"Al Junior was about 14 years old then. I said to his father, 'Hey, let’s use your son!' He said, 'Oh, man, he can’t play this shit!' But he did make the gig. He set up his kit – a cymbal, a snare drum, and a bass drum – and I kicked the thing off. And, man, that thing went off at 20 tempos!
But that was around 7:00 o'clock. And by the time Al Senior came in an hour later, at 8:00 o'clock, Al Jackson Jr. was swinging that damn band like a pro."
Future bandmates Steve Cropper and Donald "Duck" Dunn first heard Jackson playing in Mitchell's band at the Flamingo Room, and the all-white Manhattan Club. Mitchell had also hired Booker T. Jones for his band. It was Jones who suggested Jackson be brought to Stax. He said, "You guys need to know about Al." Dunn said that Jackson almost caused his wife to divorce him, because after finishing his own gig at one o'clock, he would stop by a club to hear Jackson and would get home at four or five in the morning; "He was that good!" said Dunn. It took only one session with Jackson to convince Dunn and Cropper that they had to have him. Jackson was reluctant at first. He felt he could make more money playing live than doing session work. He wanted a guaranteed regular salary to come over to Stax (although he continued to play on sessions produced by Mitchell for Hi Records). And so he became the first Stax session musician to be on a weekly salary.
Jackson became one of the most influential drummers in the history of recorded music at Stax, providing an instantly recognizable backbeat behind the label's artists, including Rufus Thomas, Carla Thomas, Eddie Floyd, Sam & Dave, Otis Redding, and blues guitarist Albert King (whose work Jackson also produced). He co-wrote "Respect" and many other Stax hits. In the 1970s, he co-wrote and played on several hits by Al Green, including "Let's Stay Together" and "I'm Still in Love with You", at Hi, and he was also a session drummer for many artists, such as Elvis Presley, Tina Turner, Bill Withers, Leon Russell, Albert King, Jerry Lee Lewis, Eric Clapton, Jean Knight, Major Lance, Ann Peebles, Rod Stewart, Shirley Brown, Donny Hathaway and Herbie Mann.
In 1975, four years after the release of their last album, Melting Pot, the members of Booker T. & the M.G.'s decided to wrap up their individual projects and devote three years to a reunion of the band. A few months later, Jackson was murdered in his home.
On September 30, 1975, Jackson was scheduled to fly from Memphis to Detroit, to produce a Major Lance session, when he supposedly heard a reminder on the radio about the Joe Frazier–Muhammad Ali fight that night. Jackson called Detroit to delay the session, saying he was going to watch the "Thrilla in Manila" on the big screen at the Mid-South Coliseum.
Though still legally married, Jackson was estranged from his wife, Barbara Jackson. In July 1975, his wife had shot him in the chest, but he decided to not press charges. He was in the process of filing for a divorce and intending to move to Atlanta, so that he could begin working with Stax singer and songwriter William Bell.
Jackson attended the screening with Eddie Floyd and Terry Manning. After the bout, he returned home to find intruders in the house. Reportedly, he was told to get down on his knees, whereupon he was fatally shot five times in the back. Around 3 a.m. on October 1, Barbara Jackson ran out in the street, yelling for help. She told police that burglars had tied her up and shot her husband when he had returned home. Police found nothing out of place in the house, and Jackson's wallet and jewelry were still on him.
The man believed to have pulled the trigger had reportedly known someone in Memphis. After robbing a bank in Florida, that person told the alleged triggerman to meet him over at Al Jackson's house. Tracked through Florida, to Memphis, and to Seattle, Washington, the suspected murderer, the boyfriend of Barbara Jackson's friend Denise LaSalle, was killed by a police officer on July 15, 1976, after an unrelated gun battle.
For recording Jackson typically used various combinations of Ludwig and Rogers drums. Two studio kits played by Jackson are on display in museums; a Ludwig kit (with a Rogers Powertone snare drum) from Stax Records in the Musicians Hall of Fame & Museum, and a Rogers kit (with a Ludwig Acrolite snare drum) from Hi Records in the Stax Museum.
According to Steve Cropper, as quoted in 'Give the Drummer Some!' by Jim Payne, a grey pearl Rogers floor tom was used in the mix 'n' match kit at Stax.