Rubinstein was born in Los Angeles, California, the son of Polish parents. His mother, Aniela (nee Mlynarska), a dancer and writer, was a Roman Catholic native of Warsaw, the daughter of conductor Emil Mlynarski. His father was Polish-born Jewish concert pianist Arthur Rubinstein. He attended El Rodeo Public School in Beverly Hills (K-2), Cours La cascade in Paris, France (1954), and St. Bernard's School (3-8) and Collegiate School (9-12) in New York City. He studied theater and music at the University of California, Los Angeles (1964-1967), and later composition at the Juilliard School in New York.
He made his Broadway acting debut in 1972 and received a Theater World Award for creating the title role in the musical Pippin, directed by Bob Fosse. In 1980 he won the Tony, Drama Desk, Los Angeles Drama Critics Circle, and Drama-Logue Awards for his portrayal of James Leeds in Mark Medoff's Children of a Lesser God, directed by Gordon Davidson.
Other Broadway appearances were in Neil Simon's Fools, directed by Mike Nichols, and Herman Wouk's The Caine Mutiny Court-Martial, which earned him a Drama Desk nomination; he replaced William Hurt as Eddie in David Rabe's Hurlyburly, replaced David Dukes in David Henry Hwang's M. Butterfly, and starred in Getting Away with Murder, by Stephen Sondheim and George Furth, directed by Jack O'Brien, and Ragtime, by Terrence McNally, Stephen Flaherty, and Lynn Ahrens. In 2014, he joined the Broadway cast of the hit revival of Pippin, directed by Diane Paulus, this time playing Pippin's father, Charlemagne; and subsequently repeated that role on the national tour throughout the United States and Japan in 2014-15.
In 1987, he made his off-Broadway debut at the Roundabout Theater as Guildenstern in Tom Stoppard's Rosencrantz and Guildenstern Are Dead, with Stephen Lang and John Wood, and subsequently performed in Urban Blight and Cabaret Verboten. In 2005, he received the Lucille Lortel Award for Best Lead Actor in a Play, as well as nominations for both the Outer Critics' and Drama League Awards, for his portrayal of George Simon in Elmer Rice's Counsellor at Law.
His appearances in regional theaters include the musicals Camelot (at various times as "Tom of Warwick", "Mordred" and "King Arthur") and South Pacific; the role of "Billy" in David Rabe's Streamers, "Ariel" in The Tempest, "Marchbanks" in Shaw's Candida, both Sergius and Bluntschli (alternating nights with Richard Thomas) in Shaw's Arms And The Man, several roles in Arnold Weinstein's Metamorphoses, directed by Paul Sills at the Mark Taper Forum in Los Angeles, Sight Unseen at L.A.'s Odyssey Theatre, The Torch-Bearers, and Our Town at the Williamstown Theater Festival, Arthur Miller's Broken Glass at Monterey Peninsula College, and Warren Smith in On a Clear Day You Can See Forever.
In 1985 he starred in Merrily We Roll Along at the La Jolla Playhouse, in a version newly re-written by Stephen Sondheim and George Furth, directed by James Lapine. He was the original Andrew Ladd III in A.R. Gurney's Love Letters at the Long Wharf Theater in New Haven, opened the play in New York off-Broadway, and later performed it on Broadway, in San Francisco, Boston, Los Angeles, Washington D.C., and on the QE II. He created the role of Molina in Kiss Of The Spider Woman, and the role of Kenneth Hoyle in Jon Robin Baitz's Three Hotels. In 1997, he played Tateh in the American premiere run of the musical Ragtime, by Terrence McNally, Stephen Flaherty, and Lynn Ahrens, at the Shubert Theater, Los Angeles, receiving both an L. A. Drama Critics Circle nomination and a Drama-Logue Award as Best Actor in a Musical, and continued in the show both in Vancouver and on Broadway. He appeared opposite Donald Sutherland in Eric-Emmanuel Schmitt's Enigmatic Variations at the Royal Alexandra Theatre in Toronto, and at the Savoy Theatre in London's West End. He played the Wizard of Oz in Wicked at the Pantages Theatre for 18 months; and co-starred in the world premiere of the musical version of the film Grumpy Old Men at the Manitoba Theatre Centre.
Rubinstein's feature films include 21 Grams, Red Dragon, Mercy, Another Stakeout, Someone to Watch Over Me, Daniel, The Boys from Brazil, Rome & Jewel, Jekyll, Kid Cop, Getting Straight, Zachariah, The Trouble with Girls, and The Car. Since 1965 he has acted in over 200 television films and series episodes including The American Clock, Mrs. Harris, Perfect Murder, Perfect Town, The Sleepwalker, The Two Mrs. Grenvilles, Movieola and Roots: The Next Generations.
He received an Emmy Award nomination for his portrayal of Jeff Maitland III in the ABC series Family, a role he played over five years; and he starred for two years with Jack Warden in the CBS series Crazy Like A Fox. He has played recurring parts on The Fosters, The Mentalist, Desperate Housewives, Parenthood, No Ordinary Family, Greek, The Wizards of Waverly Place, Dirty Sexy Money, Day Break, The Practice, Star Trek: Enterprise, The Young and the Restless, and Barbershop.
In 1987, Rubinstein made his directorial debut at the Williamstown Theater Festival, staging Aphra Behn's "The Rover", with Christopher Reeve and Kate Burton; the following season he directed the first American-cast production of Christopher Hampton's Les Liaisons Dangereuses. Off-Broadway, he directed the New York premieres of Phantasie (by Sybille Pearson) and Nightingale (by Elizabeth Diggs), and the world premiere of A.R. Gurney's The Old Boy. At the Cape Playhouse in Dennis, Massachusetts, he staged Wait Until Dark. For NYU, he directed productions of The Three Sisters and Macbeth; for UCLA, Company; and for USC, Brigadoon, Into The Woods, On The Town, City of Angels, The Most Happy Fella, and Grand Hotel.
In Los Angeles, at Interact Theatre Company, of which he has been a member since 1992, he co-directed and starred in the revival of Elmer Rice's Counsellor at Law, winning the Dramalogue and L.A. Drama Critics Circle awards in both categories, as well as Ovation Awards for Ensemble Acting and Sound Design; the production itself won 22 awards; he also directed and acted in Sondheim and Lapine's Into The Woods and A Little Night Music, and in Meredith Willson's The Music Man. He also directed Sheridan's The Rivals and Frank Loesser's Guys and Dolls. For television, he directed A Matter Of Conscience, which won the Emmy Award for Best Children's Special in 1990. He currently teaches a course in musical theater audition techniques at the University of Southern California, and directs their spring musicals.
In 2011, he provided commentary for the online broadcasting of the XIV International Tchaikovsky Competition, a classical music competition held in Moscow, Russia. Twice, in 2013 and 2015, Rubinstein appeared in a performance of The Defiant Requiem at Avery Fisher Hall, Lincoln Center.
John Rubinstein is the father of five children: Jessica Katz, Michael, Peter, Jacob, and Max.