Jarrett was born into a prosperous African-American family in Nashville, Tennessee. In 1927, when Jarrett was two years old, his father was killed in a gunfight over a woman. The family became poor, and at the age of seven Jarrett was sent to live with his grandparents on a farm outside Nashville. His step-grandfather had a violent temper and threatened him with a beating when he found he was writing song lyrics, reportedly saying "Only white boys write songs. Black boys don't write songs." At 15, Jarrett rejoined his mother and worked his way through Pearl High School in Nashville. In 1944, during World War II, Jarrett was drafted into the military, just as he was about to attend Fisk University. He ultimately returned to Fisk in the 1970s, and graduated in 1974.
In 1951, Jarrett became a disc jockey for WSOK in Nashville. He also did talent scouting for Tennessee Records. While performing at a white Nashville club called the Pink Elephant in 1955, Jarrett wrote "It's Love Baby (24 Hours a Day)", which became a #2 R&B hit for Louis Brooks and His Hi-Toppers, although the vocals were actually by Earl Gaines, a friend of Jarrett's. The song, which also charted for Hank Ballard and Ruth Brown, launched Jarrett's songwriting career.
Later in 1955, country singer Webb Pierce recorded Jarrett's "Love Love Love", which spent eight weeks at the top of the country/western music charts. In his autobiography, Jarrett tells of being stopped by a policeman outside Nashville's Hermitage Hotel because he didn't believe a black man would be invited to the 1955 BMI awards ceremony, where Jarrett was to receive an award for "Love Love Love".
In 1957, Jarrett wrote and produced "You Can Make It If You Try" for Gene Allison, which was a #3 hit on the R&B charts early in 1958 and also charted on the pop charts. The song has been covered many times since by artists such as the Rolling Stones, Buddy Guy, and Gene Vincent.
Jarrett ran or worked for a number of Nashville soul, blues, and gospel labels, including Champion, Calvert, Cherokee, Poncello, and T-Jaye. He produced for artists such as Christine Kittrell, The Fairfield Four, Larry Birdsong, Roscoe Shelton, and The Avons.
In 2004, the Country Music Hall of Fame and Museum released a compilation album entitled Night Train to Nashville, containing many tracks produced by Jarrett; it was produced to go with an exhibit of the same name at the museum. A tribute concert was held at the museum in 2005 to coincide with the publication of Jarrett's autobiography You Can Make It If You Try.
Jarrett died of liver failure in his hometown of Nashville on March 21, 2009, aged 83.