A former minor-league pitcher who played in the Pittsburgh Pirates and New York Yankees farm systems from 1972–78, Melvin began his front office career with the Yankees as an assistant scouting director, then spent nearly a decade (1987–94) as assistant general manager and director of player development of the Baltimore Orioles. In his first general manager posting, with the Texas Rangers from 1994–2001, the Rangers qualified for the postseason three times—the only times the franchise had done so to that point in its history. Melvin spent the 2002 season as a consultant for the Boston Red Sox before taking the reins in Milwaukee.
On September 26, 2002, Melvin became general manager of the Milwaukee Brewers. In 2007, he ended the franchise's 15-year streak of losing seasons when the Brewers finished above .500, although a late-season slide eliminated them from the NL Central and wild-card races. In 2008, Melvin moved the Brewers into Wild Card and, briefly, NL Central contention with his bold, mid-season pickup of pitcher CC Sabathia, but another late-season losing streak, which left the Brewers out of contention for their division title and tied for the NL Wild Card, caused Melvin and club owner Mark Attanasio to fire manager Ned Yost with 12 games left in the regular season. Melvin and Attanasio named the Brewers' third base coach Dale Sveum the interim manager for the regular season's final 12 games, and the Brewers clinched the National League Wild Card on the last day of the season, giving the team its first playoff berth since 1982.
After the 2010 season, Melvin made three separate transactions that helped the Brewers get to the playoffs in 2011. On December 6, 2010, he acquired starting pitcher, Shaun Marcum. On December 19, Zack Greinke, who won the Cy Young Award in 2009, was acquired along with shortstop Yuniesky Betancourt. On March 27, 2011, Melvin acquired outfielder Nyjer Morgan. In the same offseason, Melvin released three of the team's most ineffective starters in Jeff Suppan, Dave Bush, and Doug Davis. In 2011, the Brewers won the most games ever in franchise history with 96 victories. In so doing, the Brewers continued to do well offensively with stars Prince Fielder and Ryan Braun as well as Nyjer Morgan. Along with starters Yovani Gallardo, Randy Wolf, Chris Narveson,the newly acquired Greinke and Marcum pitched considerably more victories than the Brewers' starting pitchers the year before. The Brewers bullpen was also stellar, led by closer John Axford, who set two new franchise records with most consecutive converted saves (42) and most saves in a season (46). Furthermore, Melvin added Francisco Rodriguez before the mid-season 2011 trading deadline; Rodriguez would repeatedly pitch in the 8th inning to set up save opportunities for Axford. Along with Francisco Rodriguez, Melvin obtained veteran infielder, Jerry Hairston Jr., who would provide timely hitting to bolster an already-solid Milwaukee Brewer offense. As a result, the Milwaukee Brewers would win their division for only the second time in their history and make their fourth playoff appearance in 2011. With Nyjer Morgan's game-winning hit against the Arizona Diamondbacks in a dramatic 10th inning, the Brewers won the divisional playoff series on October 7, 2011. In the National League Championship Series, the Brewers would eventually lose to their divisional foe, the St. Louis Cardinals in six games.
On August 11, 2015, Melvin transitioned to an advisory role, leaving his GM position. Melvin is currently the President of Baseball Operations with the Milwaukee Brewers after leaving his GM role.
In 1996, Melvin received The Sporting News Executive of the Year Award, as general manager of the Rangers.
In 2011, as general manager of the Brewers, Melvin was named by Baseball America as its Major League Executive of the Year.
In 2012 Melvin was inducted into the Canadian Baseball Hall of Fame