Costabile was born in Washington, D.C. He is of Italian descent. He attended Gonzaga College High School, graduating in 1985, and Tufts University. He graduated from New York University's MFA program in acting in 1998.
Costabile has appeared on Broadway in Manhattan Theatre Club's revival of Translations in 2007, the musical Titanic in 1997, and The Tempest in 1995. In 2005, he appeared as Launce in The Two Gentlemen of Verona as part of the Public Theater's Shakespeare in the Park. For Caroline, or Change, the Drama Desk Award–winning musical by Tony Kushner and Jeanine Tesori (2003–2004), Costabile workshopped and performed in the original production and its Broadway transfer.
Costabile is best known for his recurring television roles on The Wire (as Thomas Klebanow, managing editor of The Baltimore Sun), Flight of the Conchords (as Doug), Damages (as Detective Rick Messer), Suits (as Daniel Hardman, former senior partner of the law firm of Pearson Hardman), and Breaking Bad (as Gale Boetticher). He is currently starring as Mike 'Wags' Wagner in the Showtime production Billions.
Costabile also appeared in season six of The Office, as a banker sent from Sabre to evaluate Dunder Mifflin, and in the House episode "Changes" as a limousine driver. During the second series of Franklin & Bash, he guest-starred as a senior partner of the Infeld Daniels New York office, investigating Franklin and Bash because of a complaint filed against them by a former client. He had a minor guest star role in a season 3 episode of The Good Wife, playing the defendant's lawyer in a lawsuit against Lockhart and Gardner, and in a two episode story arc on BBC One's Ripper Street. Costabile also played Mr. Braddock, an attorney to the primary suspect in an episode of The Closer titled "Unknown Trouble", the first episode of season 7.
He has appeared in several television commercials, including ads for Microsoft and FedEx. In 2011, he appeared in Lie to Me as Dr. Mitch Grandon, the head doctor of a psychiatric institution in the Season 3 episode "Funhouse". In 2013, he appeared in Elementary as Danilo Gura, a custodian of a hospital in the Season 1 episode "Lesser Evils". Costabile appeared as Internal Affairs investigator Simon Boyd in the 2013 TV show Low Winter Sun. He appeared in Person of Interest in Season 1 as Samuel Gates. He appeared in The Blacklist in Season 2 as Dr. Linus Creel. He will appear in the television series Dig. He appears in Billions as Mike "Wags" Wagner.
Costabile had an important role in 2009's Solitary Man as Gary, the husband of Jenna Fischer's character. He played Jennifer Aniston's attorney in the 2010 flick The Bounty Hunter. In 2012, he appeared in the film Lincoln as Republican representative James Ashley. In 2013, he appeared in Steven Soderbergh's film Side Effects and as Professor Horstein in Runner Runner. In 2016, he appeared as the Chief in 13 Hours: The Secret Soldiers of Benghazi.
In 2012, Costabile married Eliza Baldi in Bucks County, Pennsylvania. Their daughters, Lucy Rae Costabile and Julia Grace Costabile, were born in July 2015 and March 2017, respectively.
In an April 2013 article about television, Wired magazine called Costabile "a performer who inhabits his characters so perfectly as to consistently be a fan favorite", and "one of the best things about" every television show in which he has appeared.