Smith attended Hillcrest Christian School in Jackson, Mississippi, and the University of Mississippi. He played college baseball for the Ole Miss Rebels baseball team from 2002 through 2004, and was also the backup quarterback to Eli Manning on the Ole Miss Rebels football team. During his freshman year, Smith batted .402 and was named a Freshman All-American. Smith also played for the USA National Team in the Pan-Am games while at Ole Miss.
Smith made his Major League Baseball debut with the Colorado Rockies on September 16, 2007, in a 13–0 home win over the Florida Marlins. He did not record a hit in two total at bats during the game. He got his first hit, an infield single, on September 21, 2007, in a 2–1 extra-innings road win over the San Diego Padres. His hit came off of Doug Brocail in the top of the 12th inning. Smith played in seven games during the regular season, racking up five hits in eight at bats.
Smith's successful stint during the regular season earned him a spot on the Rockies' 2007 postseason roster. In the 2007 National League Division Series against the Philadelphia Phillies, Smith had one pinch hit in two at bats. His lone hit came in the second game of the series in a 10–5 win against the Phillies. It was an infield hit in the top of the fourth inning that loaded the bases with the Rockies down, 3–2. The next batter, Colorado second baseman Kazuo Matsui, went on to hit a memorable grand slam to give the Rockies a lead that they would never relinquish. Colorado went on to sweep the Phillies in three games and then went on to sweep the Arizona Diamondbacks in four games in the 2007 National League Championship Series. Once again, Smith had one pinch hit in two at bats. The one hit was in the fourth game of the series in a 6–4 series-clinching win against the Diamondbacks. This time, Smith hit a double that came with two outs and two runners in scoring position in the bottom of the fourth inning with the Rockies down, 1–0. The double drove in both runners to give Colorado a 2–1 lead. Smith then went on to score during the inning after a Matsui single. Colorado left fielder Matt Holliday also hit a three-run home run in the fourth inning to cap off a six-run fourth inning, all with two outs.
Smith was the final out in the 2007 World Series, striking out against Boston Red Sox's closer Jonathan Papelbon. In that postseason Smith excelled, hitting above .500.
Smith hit his first career home run, a three-run shot, on May 25, 2008, in a 4–1 home win over the New York Mets. He hit the home run off of John Maine in the bottom of the fourth inning. Smith also hit his first career inside the park home run that season, which was the eighth in Colorado Rockies history.
On January 16, 2012, the Rockies traded Smith to the Oakland Athletics for Guillermo Moscoso and Josh Outman. Due to his severe splits between facing right handed and left handed pitching, Smith platooned in his two seasons in Oakland. He hit .246 with 22 home runs for the A's between 2012 and 2013.
On December 3, 2013, Smith was traded to the San Diego Padres for pitcher Luke Gregerson. On March 30, 2014, Smith hit a home run against Los Angeles Dodgers pitcher Brian Wilson in his first at-bat as a Padre. On July 2, 2014, he signed a two-year, $13 million extension. Smith finished the season batting .266 with 12 homers and 48 RBIs. He led the Padres in batting average (.266), OBP (.367) and hits (118), as well as doubles, triples and walks. During the 2014–15 offseason, the Padres traded for a trio of outfielders in Matt Kemp, Wil Myers, and Justin Upton, leaving Smith open to be traded.
On December 30, 2014, the Padres traded Smith to the Seattle Mariners in exchange for pitcher Brandon Maurer. In his first full season in Seattle, Smith hit .248 with 12 home runs while platooning for the majority of the season with Franklin Gutierrez.
On January 6, 2017, the Mariners traded Smith to the Baltimore Orioles in exchange for pitcher Yovani Gallardo. On April 7, Smith hit his first home run of the season against the Yankees in the eighth inning to secure a 5-4 win for the Orioles. On April 26, Smith collected his first walk-off of the season, a bases loaded walk against the Rays to win in the 11th inning 5-4. Smith appeared in 13 games in the month of April for the Orioles, slashing .238/.347/.476. On May 4, in an 8-3 win against the Red Sox, Smith went 4-4 with a walk. On May 9, he hit a game-tying home run late in the game against the Nationals and the Orioles would go on to win 5-4.