Starr was born in Santa Monica, California, the son of actress Jean St. James (née Pflieger) and James Schienle. Of German, British and eastern European descent, he is a Buddhist.
Starr was cast as Bill Haverchuck in the NBC sitcom Freaks and Geeks. The series received positive reviews from critics, but it suffered from low ratings and was canceled after only one season. After cancellation, the series gained a strong cult following.
After Freaks and Geeks, Starr had a number of guest appearances and one-time roles on shows such as Ed, Mysterious Ways, Providence, King of the Hill, and Normal, Ohio. He had a small recurring role during the third season of Roswell as the character Monk. In 2002, Starr appeared in the films Stealing Harvard and Cheats. He reunited with both Freaks and Geeks producer Judd Apatow and Freaks and Geeks co-star Seth Rogen when he made a guest appearance on Apatow's situational comedy Undeclared in the episode "The Perfect Date". In 2005, he appeared in the miniseries Revelations and the CBS sitcom How I Met Your Mother.
Starr co-starred in the 2007 comedy film Knocked Up, which was directed by Judd Apatow. The same year, he had small appearances in Superbad and Walk Hard: The Dewey Cox Story, both of which were produced by Apatow. Knocked Up was critically praised and was a smash at the box office.
The success of Knocked Up led to Starr having larger roles in films, such as Good Dick and Adventureland. Starr also appears in many well received short films that premiere at film festivals, such as the Sundance Film Festival.
From 2009 to 2010, Starr was a series regular on the Starz sitcom, Party Down, portraying Roman DeBeers. The series was cancelled after two seasons and there was discussions of adapting the series into film, but it never came to fruition. Since 2011, he has co-starred on the Adult Swim television series NTSF:SD:SUV::, in which he plays Sam Stern. In 2011, Starr had a major role in the film A Good Old Fashioned Orgy and also had guest appearances on television series such as Mad Love, Community and Childrens Hospital. In the latter he reprised his role of Roman DeBeers from Party Down.
Starr reunited with Freaks and Geeks and Party Down cast member Lizzy Caplan in the 2012 film Save the Date, which premiered at the Sundance Film Festival. The same year he had a supporting role in the thriller film Deep Dark Canyon, as well as guest starring on sitcoms Parks and Recreation and New Girl.
In 2013, he had a small role playing himself in This Is the End, which was directed by Seth Rogen and Evan Goldberg and had a starring role in the Kristen Bell film The Lifeguard, later working with her again in the film Veronica Mars.
He has guest starred in The Aquabats! Super Show!, Franklin & Bash, and Drunk History. In 2014 he was cast to play Bertram Gilfoyle in the HBO comedy Silicon Valley. The show garnered several Emmy award nominations for its first two seasons. As of spring 2017, Starr is reprising his role as Gilfoyle in the fourth season of the show.