Levinson was born in Baltimore, Maryland, the son of Violet "Vi" (née Krichinsky) and Irvin Levinson, who worked in the furniture and appliance business. His family was of Russian Jewish descent.
Levinson's first writing work was for variety shows such as The Marty Feldman Comedy Machine, The Lohman and Barkley Show, The Tim Conway Show, and The Carol Burnett Show. After some success as a screenwriter – notably the Mel Brooks comedies Silent Movie (1976) and High Anxiety (1977) (in which he played a bellboy) and the Oscar-nominated script (co-written by then-wife Valerie Curtin) ...And Justice for All (1979) – Levinson began his career as a director with Diner (1982), for which he had also written the script and which earned him an Oscar nomination for Best Screenplay.
Diner was the first of four films set in the Baltimore of Levinson's youth. The other three were Tin Men (1987), a story of aluminum-siding salesmen in the 1960s starring Richard Dreyfuss and Danny DeVito; the immigrant family saga Avalon (1990) featuring Elijah Wood in one of his earliest screen appearances, and Liberty Heights (1999).
His biggest hit, both critically and financially, was Rain Man (1988), a sibling drama starring Dustin Hoffman and Tom Cruise (Levinson appeared in a cameo as a doctor). The film won four Academy Awards, including Best Picture and Best Director. It also won the Golden Bear at the 39th Berlin International Film Festival.
Another of his notable films is the popular period baseball drama The Natural (1984), starring Robert Redford. Redford would later direct Quiz Show (1994) and cast Levinson as television personality Dave Garroway. Levinson also directed the classic war comedy Good Morning, Vietnam (1987), starring Robin Williams, with whom he later collaborated on the fantasy Toys (1992) and the political comedy Man of the Year (2006). Levinson also directed the critically acclaimed historical crime drama Bugsy (1991), which starred Warren Beatty and was nominated for ten Academy Awards.
He directed Dustin Hoffman again in Wag the Dog (1997), a political comedy co-starring Robert De Niro about a war staged in a film studio (Levinson had been an uncredited co-writer on Hoffman's 1982 hit comedy Tootsie). The film won the Silver Bear – Special Jury Prize at the 48th Berlin International Film Festival.
Levinson partnered with producer Mark Johnson to form the film production company Baltimore Pictures. The two parted ways in 1994. Levinson has been a producer or executive producer for such major productions as The Perfect Storm (2000), directed by Wolfgang Petersen; Analyze That (2002), starring De Niro as a neurotic mob boss and Billy Crystal as his therapist, and Possession (2002), based on the best-selling novel by A. S. Byatt.
He has a television production company with Tom Fontana (The Levinson/Fontana Company) and served as executive producer for a number of series, including Homicide: Life on the Street (which ran on NBC from 1993 to 1999) and the HBO prison drama Oz. Levinson also played an uncredited main role as a judge in the short-lived TV series The Jury.
Levinson published his first novel, Sixty-Six (ISBN 0-7679-1533-X), in 2003. Like several of his films, it is semi-autobiographical and set in Baltimore in the 1960s. He directed two webisodes of the American Express ads "The Adventures of Seinfeld and Superman". In 2004, Levinson was the recipient of the Austin Film Festival's Distinguished Screenwriter Award. Levinson directed a documentary PoliWood about the 2008 Democratic and Republican National Conventions. The documentary, produced by Tim Daly, Robin Bronk and Robert E. Baruc, had its premiere at the 2009 Tribeca Film Festival.
Levinson is developing a film based on Whitey Bulger, the Boston crime boss. The film Black Mass (script by Jim Sheridan, Jez Butterworth, and Russell Gewirtz) is based on the book by Dick Lehr and Gerard O'Neill, and is said to be the "true story of Billy Bulger, Whitey Bulger, FBI agent John Connelly and the FBI's witness protection program that was created by J. Edgar Hoover."
In September 2013, Levinson was set to direct the film titled Rock the Kasbah, written by Mitch Glazer. Bruce Willis, Shia LaBeouf, Bill Murray and Kate Hudson will star in the film. He has also finished production on The Humbling (2014), starring Al Pacino.
In 2010 Levinson received the Laurel Award for Screenwriting Achievement, which is the lifetime achievement award from the Writers Guild of America.