Gurley was born in Detroit, Michigan.
At the age of nineteen Gurley took up the guitar and began practicing long hours while listening to old Lightnin' Hopkins records. He never had a guitar lesson, preferring to learn by ear. He spent four years at Detroit's Catholic Brothers of the Holy Cross, studying to be a priest.
In 1962, he moved to San Francisco with his wife Nancy and became part of the coffee-house circuit, playing in the folk and country blues tradition. For a time, he played with J. P. Pickens and the Progressive Bluegrass Boys.
In the summer of 1965, Chet Helms brought Gurley to 1090 Page Street to meet Peter Albin and Sam Andrew of Big Brother and the Holding Company, and shortly thereafter, he joined the band. His fearlessly wild guitar playing made the band's reputation for "far-out" psychedelic experimentation. He said it developed from his admiration of John Coltrane's barrier-breaking saxophone solos.
A few weeks after Janis Joplin joined the band, Gurley began having an affair with her. Once Gurley and Joplin became involved, he moved out of the apartment he shared with his wife and moved in with Joplin. According to Joplin, that arrangement ended the day James' wife Nancy Gurley came barging through the front door of Joplin's apartment. "What an embarrassing situation," Joplin told her friend Jim Langdon later. "His old lady comes marching into my bedroom with the kid and the dog and confronts us." Gurley continued his affair with Joplin for a while, but eventually returned to Nancy, who forgave both him and Joplin, with whom she had a close friendship. In 1966, the members of Big Brother, along with their wives and children, all moved into a single house in Lagunitas, California.
With Joplin's departure, Big Brother and the Holding Company briefly disbanded in 1968, but a new lineup including Gurley and the other three original members (Sam Andrew, Peter Albin and Dave Getz) reunited from 1969 to 1972.
In 1969, Nancy Gurley died of a heroin overdose. Gurley was charged with murder for injecting the drugs, and spent two years fighting the charges before being sentenced to probation. He remarried and had another child in 1972.
In 1978 Gurley started a band with his son Hongo on drums, featuring then girlfriend "Red" Robin Reed on rhythm guitar/lead vocals and Jim Holt ("Jerome") on lead psychedelic rock saxophone and vocals. As he did in later incarnations of Big Brother and the Holding Company, James added his distinctive bass lines and vocals; he also engineered and mixed the band's recordings. The band became Red Robin and the Worms, a psychedelic-edged early new wave, punk and reggae-flavored group, playing all originals. Later Dennis Franklin joined as lead guitar, contributing new songs and singing some lead vocals. Dennis later left to pursue a solo career. After auditioning guitarists, Mitch McKendry aka Mitch Master was asked to join and take over lead guitar.
Big Brother and the Holding Company again reunited in 1987 with original members Gurley, San Andrews, Peter Albin and Dave Getz.
In May 1997, Gurley stopped touring with Big Brother to devote his full attention to other collaborative and solo projects. The first offering was a CD entitled Saint James – Pipe Dreams which included two members of the Worms, Mitch McKendry and Hongo Gurley. St James – Pipe Dreams was released in January 2000.
After finding a long-forgotten live 4-track recording of Janis Joplin playing acoustic guitar and singing in a Bay Area venue, Gurley laboriously added drums, bass and whatever other instrumentation he deemed the songs needed and released it as a CD This is Janis Joplin. Shortly afterwards, he was contacted by someone speaking for Joplin's estate and told to desist.
Gurley, with fellow members of Big Brother, played at the induction ceremony for Joplin at the Rock and Roll Hall of Fame in 1995, although he regretted that the band members were not inducted themselves.. He also reunited with the band in the Bay Area at a concert for Chet Helms.
Gurley recorded and appeared with new age drummer and percussionist Muruga Booker for many years, as well as his last recording, called Big Huge. which was recorded just 2 months prior to his death.
Gurley died on December 20, 2009, from a heart attack at his home in Palm Desert, California, two days before his 70th birthday. His wife, Margaret, survived him, as did his two sons, Django and Hongo.