In high school, Zimmer was a successful multi-sport athlete who earned all-conference honors in football (1972–73), baseball, and wrestling at Lockport Township High School in Lockport, Illinois. He enrolled at Illinois State University, majoring in physical education. He played quarterback in 1974, but he redshirted the following year and moved to linebacker in 1976 after breaking his thumb. A neck injury prematurely ended his playing career that same year. His college teammates were positively influenced by his constantly "up" attitude, especially during those hot August pre-season double workouts. He backed up quarterback Bob Lopez in 1974, who later went on to set many of Illinois State's passing records.
Zimmer's first coaching job was as a part-time defensive assistant at the University of Missouri from 1979–1980. He then coached at Weber State College from 1981-1988, serving as the inside linebackers coach from 1981–84, the defensive backs coach from 1985–88, and the defensive coordinator from 1983–88. From 1989-1993, Zimmer served as the defensive coordinator and defensive backs coach for the Washington State Cougars. In 1993, the Cougars defense ranked eighth in the nation in total defense and second in rushing defense.
He joined the Dallas Cowboys in 1994 as an assistant coach of the nickel defense under Barry Switzer. He was promoted to defensive backs coach in 1995 and served in that capacity before being promoted to defensive coordinator in 2000. The 2003 Dallas Cowboys defense gave up the fewest yards in the NFL while running an aggressive, speedy 4-3 defense. Despite the Cowboys' problems over the years, Zimmer survived several coaching changes (Switzer, Chan Gailey, Dave Campo, Bill Parcells) and was rumored to have been a candidate for the head coaching job at the University of Nebraska (circa 2003). In 2005, he implemented the 3-4 defense favored by head coach Bill Parcells, although Zimmer had no prior experience with it.
When Bobby Petrino was hired to coach the Atlanta Falcons early in 2007, Mike Zimmer agreed to become the new defensive coordinator in Atlanta. Zimmer coached in Atlanta for only one season after Petrino left the Falcons for the University of Arkansas after thirteen games. Zimmer has been very outspoken against Petrino since the events of Petrino's unexpected departure from Atlanta in 2007.
Zimmer was named the defensive coordinator for the Cincinnati Bengals on January 15, 2008. In 2009, Zimmer earned NFL Assistant Coach of the Year honors from Pro Football Weekly/Pro Football Writers and from CBSSports.com, after guiding the Bengals to the 4th ranked defense in the league. In 2011, the Bengals finished with the 7th ranked defense in total yards and 9th ranked defense in points allowed.
In 2012, the Bengals finished with the 6th ranked defense in total yards and 8th ranked defense in points allowed, prompting the Cleveland Browns to interview Zimmer for their head coaching vacancy. The Browns eventually hired former offensive coordinator Rob Chudzinski on January 11, 2013.
On January 15, 2014, Zimmer earned his first head coaching position when the Minnesota Vikings hired him to replace Leslie Frazier. Zimmer is the 9th head coach in Viking history.
Zimmer earned his first win as the Vikings Head Coach on September 7, 2014 against the St. Louis Rams with a score of 34-6.
He ended his first year with a record of 7 wins and 9 losses, an improvement over the 2013 record of 5-10-1, and the best record for a first year head coach in the Minnesota Vikings franchise since Dennis Green in 1992. In 2015, Zimmer improved on his 2014 record by ending the season with a record of 11-5 and winning the NFC North, ending Green Bay's streak of four consecutive division titles and giving the Vikings their first since 2009. At TCF Bank Stadium, the Vikings lost the wild-card playoff round to the Seattle Seahawks 10–9 while playing in subzero temperatures and the third-coldest game in NFL history.
Zimmer suffered a detached retina during the October 31, 2016 Monday Night Football game against the Chicago Bears. On a windy night at Soldier Field, the edge of Zimmer's playcard, which was laminated, blew into his eye. Zimmer underwent emergency eye surgery on November 30, 2016 forcing him to miss the Vikings' Thursday Night Football game against the Dallas Cowboys. Special teams coach Mike Priefer served as interim coach for the game. Zimmer finished the 2016 NFL season with a record of 8-8, but the Vikings failed to make the playoffs.
Before week 13 against the Dallas Cowboys Mike Zimmer had emergency eye surgery and was unable to coach the Vikings that week.
Zimmer is the son of former NFL player and coach, Bill Zimmer, who died on August 12, 2015 at age 84.
Zimmer's wife, Vikki, died unexpectedly at their Ohio home on October 8, 2009. Three days later, Zimmer was awarded the game ball by Bengals head coach Marvin Lewis after the team's 17–14 victory over the Baltimore Ravens.
Zimmer has three children: Adam, Corri, and Marki. Adam Zimmer is the current linebackers coach for the Vikings. Adam also served as an assistant linebackers coach with the New Orleans Saints during their Super Bowl-winning season.
Zimmer owns a hunting lodge in Northern Kentucky.