Oppenheimer was born in Yonkers, New York, attended Cornell University for one year in 1948, spent less than one semester at the University of Chicago, and in 1950 enrolled at Black Mountain College in North Carolina. At Black Mountain, he studied with Paul Goodman and poet Charles Olson, became friends with Fielding Dawson and Ed Dorn, and worked in the school's print shop.
In his earliest poetry, Oppenheimer shows clearly the influence of William Carlos Williams, but he soon developed his own style. While at Black Mountain, Oppenheimer met and married his first wife, Rena Furlong. He left the school in January 1953 without taking a degree, eventually settling in New York and working in a print shop while continuing to write poetry.
His first publications were The Dancer (1951), as Jargon, no. 2, 1951, by The Sad Devil Press/Black Mountain College; The Dutiful Son (1956) by Jonathan Williams's Jargon Society, reprinted by LeRoi Jones's Totem Press in 1961, The Love Bit and Other Poems (1962), again with Totem. His satiric Western drama The Great American Desert was the first play produced by Robert Nichols, directed by Lawrence Kornfeld, who had been with the Living Theatre, at the Judson Poets' Theatre. It opened on November 18, 1961.
Oppenheimer's poetry has been collected in two volumes: Robert J. Bertholf (editor, introduction), Collected Later Poems of Joel Oppenheimer, with eleven drawings by John Dobbs, The Poetry Collection, 1997 and Names & Local Habitations (Selected Earlier Poems 1951-1972), editor Jonathan Williams, The Jargon Society, 1988.
He also published two nonfiction works, The Wrong Season, Bobbs-Merrill 1973, about the New York Mets, and Marilyn Lives, Delilah, 1984, on Marilyn Monroe. Drawing from Life, posthumously published in 1997, gathered 92 columns written for the Village Voice. Library Journal wrote that Drawing from Life "emphasizes several favorite themes: baseball, politics, and the role of the changing seasons in our lives".
Oppenheimer died at 58 of lung cancer in Henniker, New Hampshire on October 11, 1988.
Don’t Touch the Poet: The Life and Times of Joel Oppenheimer, by Lyman Gilmore, was published by Talisman Press in 1998.