Franz was born in Maywood, Illinois, the son of German immigrants Eleanor, a postal worker, and Franz Schlachta, who was a baker and postal worker. He has two older sisters, Heidi (born 1935) and Marlene (born 1938). Franz is a graduate of Proviso East High School (in Maywood), Wilbur Wright College and Southern Illinois University Carbondale. After graduating from college, Franz was drafted into the United States Army. He served eleven months with the 82nd Airborne Division and the 101st Airborne Division in Vietnam.
Franz began his acting career at Chicago's Organic Theater Company. Although he has in the past performed Shakespeare, his appearance led to his being typecast early in his career as a police officer. (By Franz's own count, the character of Andy Sipowicz was his 28th role as a police officer). He has also guest starred in shows such as The A-Team. Other major roles were on the television series Hill Street Blues in which he played two characters over the run of the show. Franz first played the role of Detective Sal Benedetto, a corrupt cop in the 1983 season, who later kills himself. Due to his popularity with fans, he returned in 1985 as Lt. Norm Buntz, remaining until the show's end in 1987. He also starred in the short-lived Beverly Hills Buntz as the same character.
During the late 1970s and early 1980s, Franz worked regularly with directors Brian De Palma and Robert Altman. He appeared in three of Altman's films from this period, and five of De Palma's, most prominently as a low budget movie director in Body Double (1984).
Franz went on to win four Emmy Awards for his portrayal of Andy Sipowicz on NYPD Blue. The character of Sipowicz was ranked #23 on Bravo's 100 Greatest TV Characters list.
In 1994 Franz made a cameo appearance as himself in The Simpsons episode "Homer Badman" - when Homer is accused of sexually harassing a babysitter, the case becomes tabloid fodder, generating an exploitative television movie, Homer S.: Portrait of an Ass-Grabber, in which Franz portrays Homer.
On May 11, 2001, Franz was a contestant on a celebrity edition of the hit television game show Who Wants to Be a Millionaire, winning $250,000 for his charity.
Franz also was a commercial spokesman for Nextel in the early 2000s. The concept was that Franz "refused" to do the commercials, saying they were not something he did.
He starred as Earl, the abusive husband, in the Dixie Chicks' music video "Goodbye Earl", as airport police captain Carmine Lorenzo in the 1990 film Die Hard 2 and as Nathaniel Messinger in the 1998 film, City of Angels. It would be his last film as he spent the final years of his career on NYPD Blue.
Franz has stayed out of the acting spotlight since 2005 to focus on his private life. He has told Access Hollywood he would be interested in returning to acting if given the right opportunity. He made a special appearance at the 2016 Primetime Emmy Awards.
In 1995 Franz married Joanie Zeck, whom he met in 1982. He is the stepfather of Zeck's two daughters, Tricia and Krista, from a previous marriage.