Carman was born in Trenton, New Jersey. As a child, he played drums in his mother's band, then as a teen, he formed his own group. Around the age of 20, he moved to California.
While attending an Andraé Crouch concert, Carman became a born again Christian, and embraced evangelicalism. In 1980, he made a custom album titled God's Not Finished with Me. The following year, he was invited by Bill Gaither to tour with The Bill Gaither Trio. After relocating to Tulsa, Oklahoma, he released a moderately successful eponymous debut album (later issued as Some-o-Dat) in 1982—which contained mostly novelty songs. Then, with the release of the live album Sunday's On The Way in 1983, a string of CCM chart successes started, beginning with the title song. As he continued his music career, Carman established the nonprofit organization Carman Ministries. With the 1985 release of The Champion came his first number-one song, of the same name. Another live album, Radically Saved, came in 1988. Then 1989 brought Carman his first number one album, Revival in the Land.
Between 1987 and 1989, he was named Charisma magazine's readers' choice for favorite male vocalist. In 1990 and 1992, Billboard recognized Carman's influence in Christian music by naming him the Contemporary Christian Artist of the Year, and in 1995, he recorded his first album in Spanish, Lo Mejor, which includes some of the already recorded songs such as "Who's in the House" and "The River". In 2000, the album Heart of a Champion came out with 30 of his best songs, including some new ones. In 2007, he updated his catalog with Instrument Of Praise.
Carman has received 15 gold and platinum albums and videos and has sold over 10 million records. He holds the world record for the largest single Christian concert in history. He was the main act in August 1993 in Johannesburg, South Africa with more than 50,000 in attendance, and he performed at Texas Stadium with 71,132 attendees.
Beyond music, he participates in various television productions and interview duties as a host for both the Trinity Broadcasting Network in general and its flagship program Praise the Lord. In 2001, Carman starred in the film Carman: The Champion.
In November 2011, Carman was involved in an auto accident. He was in the passenger seat when the driver made a left into the church parking lot in front of a crowd of fans waiting to get into the concert. A truck from oncoming traffic struck Carman's side at 50 miles per hour (80 km/h). Carman was able to walk into the church, attempting to continue with the scheduled concert, but collapsed soon after. He was taken to a hospital just outside Houston, where he was examined and x-rayed. No broken bones or spinal injuries were found. However, internal injuries were found that required surgery.
In a photo post made on both his official and personal Facebook accounts on February 14, 2013, Carman revealed that he had been diagnosed with myeloma and given a prognosis of only three to four more years to live.
In late March 2013, Carman announced a Kickstarter campaign for a new album and music video. A short time later, he announced an upcoming 60-city tour, as well, additionally noting that the online fundraising campaign had raised more than $230,000 within several weeks. As of December 2013, Carman had been in the hospital receiving chemotherapy treatments. By early 2014 however, Carman announced that medical tests had indicated that his body was free of cancer. He planned to continue preparing for his Live Across America album and tour.
In January 2015 Carman attributed his successful recovery to the "commitment" he had made to his Facebook and Kickstarter supporters to begin his "No Plan B Tour" as soon as he regained his health. As of the fall of 2015, Carman was touring the eastern United States.