A three-year varsity letterman at Amelia High School and a Division I All-Fort Ancient Valley Conference honoree, Smith also pitched five years for the select American Amateur Baseball Congress Midland team in Cincinnati. Even through his labrum surgery he was dedicated to return to baseball. In college, he was redshirted and played three years in the Horizon League. Ultimately, Smith became Wright State's closer his junior and senior year, as he gained 4–6 mph on his fastball after changing his delivery. He had 13 saves his senior season and a 0.98 ERA. In three seasons Smith posted 22 saves, 145 strikeouts, and 39 walks. In 2005, he was the WSU team MVP and in 2006 he was awarded second team All-Mideast Region, first team Horizon League, Horizon League Pitcher of the Year and WSU Most Valuable Male Athlete.
In the late summer of 2004, Smith played summer ball with the NECBL with the North Adams Steeplecats. In 2005, Smith briefly pitched for the Rockville Express, a team in the Cal Ripken, Sr. Collegiate Baseball League In 2005, he made nine appearances for the Express, posting a 4.66 ERA, a 1–0 record and earning two saves. Later in the summer of 2005, Smith also pitched for the Edenton Steamers in the Coastal Plain League. In 16 regular season appearances, totaling 17.1 innings, he tallied nine saves with a 2.08 ERA, .197 batting average against, and struck out 19 batters. He also recorded the final out of the 2005 Petitt Cup tournament, on a comebacker, to give Edenton its second consecutive league championship. Less than two years later, Smith became the first Steamers' alumnus to appear in the majors.
Smith first began his career with Brooklyn (A) of the New York–Penn League by allowing two runs, one earned, in one inning of work against the Staten Island Yankees on June 21. He was named one of the top 20 prospects in the New York–Penn League by Baseball America. When Duaner Sánchez was hurt during a taxi cab incident in 2006, Mets general manager Omar Minaya considered recalling Smith to the Majors. Instead, the Mets traded for Roberto Hernández and Óliver Pérez to give Smith more time in the minors. On March 24, 2007, after a strong spring training, it was announced Smith would be on the Mets' 25-man roster. On April 1, 2007, Smith made his major league debut. He struck out one (Preston Wilson) and walked one (Albert Pujols). He also gave up a single to David Eckstein. Smith earned his first major league win on April 24, 2007 after coming on in relief in the 12th inning. Smith enjoyed much success in the beginning of 2007, but began to tire and was sent down to the minors at mid year. Smith was with the Mets for all of the 2008 season, enjoying varied success. He went 6–3 in 63.3 innings with a 3.55 ERA.
On December 10, 2008, Smith was traded by the Mets to the Cleveland Indians as part of a twelve-player, three-team deal. On January 18, 2013, the Cleveland Indians announced they had avoided arbitration with Smith, signing him to a one-year contract worth $3.15 million.
Smith became a free agent following the 2013 season, but expressed interest in remaining with the Tribe.
On November 24, 2013, Smith reportedly agreed to a three-year contract for $15 million with the Los Angeles Angels of Anaheim, pending the completion of a physical. On November 27, 2013, the Angels confirmed that the team and Smith agreed to the terms. On April 25, 2014, Smith was named the Angels' new closer after multiple struggles by former closer Ernesto Frieri. After the Angels traded for San Diego Padres' All-Star closer Huston Street on July 18, 2014, Smith was moved to an eighth-inning setup role. Smith finished with 15 saves in 76 games to the tune of a 1.81 ERA. Smith's numbers in 2015 went up as his ERA and WHIP from the previous season. On June 8, 2016, Smith went on the disabled list with a hamstring injury.
On August 1, 2016, the Angels traded Smith to the Chicago Cubs for prospect Jesus Castillo. In 16 more appearances with the Cubs to finish 2016, Smith had a 1-1 record and a 2.51 ERA. Overall in 2016, combined with both teams he played for, Smith made 54 total appearances with a 1-4 record and a 3.82 ERA. Smith was a member of the Cubs' 2016 World Series championship team, but did not make any postseason appearances.
On February 9, 2017, Smith signed a one-year, $3 million contract with the Toronto Blue Jays. Smith became the Blue Jays setup man early in the season, after Jason Grilli struggled in the role and Joe Biagini was moved to the starting rotation. He was placed on the 10-day disabled list on June 19 with shoulder inflammation.
On July 31, 2017, the Blue Jays traded Smith to the Cleveland Indians for Thomas Pannone and Samad Taylor.
Smith's pitching style is different from most. According to hitters, he comes at the batter from third base. His release point is about 8:30 on the analog clock, lower than side armed, and higher than submarine. Smith's fastball usually is at 88–90 mph and has been known to hit 93 mph. His fastball can even be considered a sinker because of its hard downward movement. He routinely gets ground balls because of this. Smith also throws a slider and a changeup.
Smith is married to Allie LaForce of CBS. In 2012, Smith's mother was diagnosed with an inherited disease known as Huntington's disease; the condition leads to the death of nerve cells in the brain. Smith has a 50 percent chance of inheriting the condition and says that he will be tested for it before he has children.