Smith was born in Gloucester, Massachusetts. His father was Benjamin A. Smith II, a U.S. Senator.
Smith played ice hockey at Harvard University and graduated in 1968. For three of his four years at Harvard, Ben Smith was on the men’s hockey team. His coach was the legendary Cooney Weiland.
Upon graduation from Harvard, Weiland advised Smith to make coaching a career. In autumn 1968, Smith served as an assistant at the University of Massachusetts Amherst. After several years of coaching high school hockey in Gloucester, Massachusetts, Smith became an assistant with the Yale Bulldogs.
Smith left Yale to become an assistant with the Boston University Terriers. At BU, Smith worked with Jack Parker, who played for BU while Smith played at Harvard.
Smith’s time at BU would represent some of the first success in his coaching career. After helping to lead the Terriers to the NCAA Frozen Four in 1990, he accepted the head coaching job for the Dartmouth Big Green. In his first and only season with Dartmouth, he had 1 win, 24 losses and 3 ties. His only victory was against the Northeastern Huskies, the team he would coach the following season.
Smith became the Northeastern coach in 1991, inheriting a team that had 8 wins, 25 losses and 2 ties in the 1990–91 season.
While he was an assistant coach at Boston University, he took a year off from BU to assume a role as the assistant coach of the U.S. men’s hockey team in Ice hockey at the 1988 Winter Olympics in Calgary. Ten years later, Smith would return to the Olympics as the coach of the first ever US women’s team. Smith coached the first three women’s Olympic teams and won a gold (1998), silver (2002) and bronze (2006) medal. In 2009, the 1998 U.S. Olympic Women's Ice Hockey Team was inducted into the United States Hockey Hall of Fame.
Smith is known for his self-deprecating style and the "amusing and often puzzling sayings" that he uses to motivate his players, such as "The hay is in the barn, ladies".