Regarded as the most important recording engineer of jazz by some observers, Van Gelder recorded several thousand jazz sessions, including many recognized as classics, in a career which spanned more than half a century. Van Gelder recorded many of the great names in the genre, including John Coltrane, Miles Davis, Thelonious Monk, Sonny Rollins, Art Blakey, Joe Henderson, Freddie Hubbard, Wayne Shorter, and Horace Silver, among many others. He worked with many record companies but was most closely associated with Blue Note Records. The New York Times wrote his work included "acknowledged classics like Coltrane's A Love Supreme, Davis's Walkin', Herbie Hancock's Maiden Voyage, Sonny Rollins's Saxophone Colossus, and Horace Silver's Song for My Father."
Van Gelder was born in Jersey City, New Jersey. His parents, Louis Van Gelder and the former Sarah Cohen, ran a women’s clothing store in Passaic. His interest in microphones and electronics can be traced to a youthful enthusiasm for amateur radio. A longtime jazz fan (his uncle, for whom Rudy was named, had been the drummer for Ted Lewis's band in the mid-1930s), Van Gelder took lessons on the trumpet. Van Gelder trained as an optometrist at the Pennsylvania College of Optometry, in Philadelphia, because he did not think he could earn a living as a recording engineer. From 1943, after graduating, Van Gelder had an optometry practice in Teaneck, New Jersey, and recorded local musicians in the evenings who wanted 78-rpm recordings of their work. He became a full-time recording engineer in 1959. From 1946, Van Gelder recorded in his parents' house in Hackensack, New Jersey, in which a control room was built adjacent to the living room, which served as the musicians' performing area. The dry acoustics of this working space were partly responsible for Van Gelder's inimitable recording aesthetic.
"When I first started, I was interested in improving the quality of the playback equipment I had," Van Gelder commented in 2005; "I never was really happy with what I heard. I always assumed the records made by the big companies sounded better than what I could reproduce. So that's how I got interested in the process. I acquired everything I could to play back audio: speakers, turntables, amplifiers". One of Van Gelder's friends, the baritone saxophonist Gil Mellé, introduced him to Alfred Lion, a producer for Blue Note Records, in 1953.
Within a few years Van Gelder was in demand by many other independent labels based around New York City, such as Prestige Records. Bob Weinstock, owner of Prestige, recalled in 1999, "Rudy was very much an asset. His rates were fair and he didn’t waste time. When you arrived at his studio he was prepared. His equipment was always ahead of its time and he was a genius when it came to recording". According to a JazzTimes article in August 2016, "jazz lore has formed the brands into a yin and yang of sorts: The Blue Note albums involved more original music, with rehearsal and the stringent, consistent oversight of Lion; Weinstock was more nonchalant, organizing what were essentially blowing sessions for some of the best musicians in jazz history". Van Gelder said in 2012, "Alfred was rigid about how he wanted Blue Note records to sound. But Bob Weinstock of Prestige was more easygoing, so I'd experiment on his dates and use what I learned on the Blue Note sessions". He also worked for Savoy Records in this period, among others. "To accommodate everyone, I assigned different days of the week to different labels".
In the 1950s Van Gelder also performed engineering and mastering for the classical label Vox Records. Thelonious Monk composed and recorded a tribute to Van Gelder entitled "Hackensack".
Until the late 1950s Van Gelder worked during the day as an optometrist. In the summer of 1959, Van Gelder moved his operations to a larger studio in Englewood Cliffs, a few miles southeast of the original location, and left his day job in favor of recording full-time. The new studio's design was inspired by the work of Frank Lloyd Wright and bore some resemblance to a chapel, with high ceilings and fine acoustics. An obituarist in the London Daily Telegraph wrote of "Van Gelder's extreme fastidiousness" as an engineer, and his insistence on "no food or drink in the studio, and on no account was anyone to touch a microphone. He himself always wore gloves when handling equipment".
It was in Englewood Cliffs that John Coltrane recorded the album A Love Supreme for Impulse! Records in 1964. Other labels, such as Verve Records, made use of the new facility, while Blue Note and Prestige continued their associations with Van Gelder for several years.
In 1967, Alfred Lion retired from running Blue Note, and the company's owners, Liberty Records (from 1965), began to use other engineers more regularly. Prestige, too, had started to use other studios a few years earlier. Van Gelder remained active in music recording, working as the engineer for most of Creed Taylor's CTI Records releases, a series of proto-smooth jazz albums that were financially successful, but not always well received by critics.
Though his output slowed, Van Gelder remained active as a recording engineer into the new century. In the late 1990s he worked as a recording engineer for some of the songs featured on the soundtrack to the TV Show Cowboy Bebop. From 1999, he remastered the analog Blue Note recordings he made several decades earlier into 24-bit digital recordings in its RVG Edition series and also a similar series of remasters of some of the Prestige albums he recorded for its current owners, Concord Records. He was positive about the switch from analog to digital technology. He told Audio magazine in 1995:
The biggest distorter is the LP itself. I've made thousands of LP masters. I used to make 17 a day, with two lathes going simultaneously, and I'm glad to see the LP go. As far as I'm concerned, good riddance. It was a constant battle to try to make that music sound the way it should. It was never any good. And if people don't like what they hear in digital, they should blame the engineer who did it. Blame the mastering house. Blame the mixing engineer. That's why some digital recordings sound terrible, and I'm not denying that they do, but don't blame the medium.
Van Gelder resided in Englewood Cliffs into this century. He died at his Engelwood Cliffs home on August 25, 2016.
Van Gelder was secretive about his recording methods, leaving fans and critics to speculate about his techniques. His recording techniques are often admired for their transparency, warmth and presence. Richard Cook called Van Gelder's characteristic method of recording and mixing the piano "as distinctive as the pianists' playing" itself. Blue Note president and producer Alfred Lion criticized Van Gelder for what Lion felt was his occasional overuse of reverb, and would jokingly refer to this trait as a "Rudy special" on tape boxes. Despite his prominence in recording jazz, some artists avoided Van Gelder's studio. The bassist and composer Charles Mingus refused to record with him. Taking Leonard Feather's "blindfold test" in 1960, he said that Van Gelder "tries to change people’s tones. I’ve seen him do it; I’ve seen him do it; I’ve seen him take Thad Jones and the way he sets him up at the mike, he can change the whole sound. That’s why I never go to him; he ruined my bass sound". IIn 2009 Van Gelder was named a fellow of the Audio Engineering Society (AES). In 2013, he received the society's most prestigious award, the AES Gold Medal.
In 2009 he was named a Jazz Master by the National Endowment for the Arts.
In 2012 he received the Grammy Trustees Award.