He grew up in Warradale, South Australia, and attended Brighton High School in Adelaide. He later obtained a teaching degree and whilst at Sturt Teachers' College took up drama.
In 1984, Sweet had his first major role as Donald Bradman in the Network Ten miniseries Bodyline, the story of the 1932–33 Test cricket series between England and Australia.
The award-winning 1987 Australian TV movie The Great Bookie Robbery was a gritty true-crime story seeing Sweet's character Chico White playing the inside man trying to infiltrate the close-knit bank robbers.
In 1994, he appeared in The Battlers.
Between 1990 and 1996, Sweet starred in the drama series Police Rescue as Sgt. Steve "Mickey" McClintock. This role was a popular role for Sweet, and lead him to win several major television awards, including the Australian Film Institute's Award for Best Actor in a Leading Role in a Television Drama for his performance (in 1991 and 1992), the Variety Club Heart Award for TV Actor of the Year (1993), and two TV Week Silver Logie Awards for Most Popular Actor and Most Outstanding Actor (1992, 1994).
Because of his popularity in Police Rescue, rumours swirled in 1993 he was going to switch to the Seven Network for "a seven figure contract" but this never eventuated.
Between 1997 and 1999, Sweet starred in Big Sky as Chris Manning, and in 1999, starred in Dog's Head Bay as Alex. In 2001, he appeared in a fourth series episode of the British comedy-drama Cold Feet. In 2002, he took on the continuing role of Inspector Luke Harris in the police drama series Stingers until the show's conclusion in 2004. He then went on to star in the Australian Broadcasting Corporation's "Dog's Head Bay".
In 2006, Sweet was a contestant on Dancing with the Stars and made it to the final seven. His partner was dancer Eliza Campagna. In 2007, Sweet appeared on several shows. He starred in the SBS miniseries The Circuit (starring as Magistrate Peter Lockhart), he co-hosted Channel Nine's Things To Try Before You Die and starred in the ABC television series Rain Shadow with Rachel Ward. In 2008, he narrated the Australian observational documentary TV series Sudden Impact, broadcast on the Nine Network.
In 2009, he had a guest role in Channel Nine's Rescue: Special Ops and starred in season two of The Circuit. He had a role as Gunnery Sergeant Elmo "Gunny" Haney in The Pacific, which aired in 2010 in Australia and the US in 2010, he played Superintendent Jack Finchin in the short-lived Nine Network police drama Cops L.A.C. In 2011, he had a role on Rescue: Special Ops as Shane Gallagher, the father of Dean and Chase Gallagher. Since 2012, Sweet has starred in the Nine Networks drama/comedy House Husbands.
Though most recognised for his television roles, Sweet has appeared in numerous stage productions. These include David Williamson's The Club, and in 2001, Tony McNamara's play The Recruit for the Melbourne Theatre Company. He has appeared in the Frankie Valli-tribute musical Oh, What a Night. In 2013, he performed in Trevor Ashley's musical comedy Little Orphan trAshley with Rhonda Burchmore.
Sweet dabbled in the Australian music scene with limited success in the early 1990s. Sweet released a cover of Billy Thorpe's Most People I Know (Think That I'm Crazy) in 1994 through Polygram Records Australia. The single peaked at #52 on the ARIA singles chart, and spent 7 weeks in the top 100.
Sweet has won several awards in his acting career. He has won two Logie Awards, one in 1982 for his role in The Sullivans (won the Most Popular New Talent) and one in 1994 for his role on Police Rescue (won the Most Popular Actor).
At the 1992 and 1996 Logies, he was nominated for Most Popular Actor for his role in Police Rescue, and in 2003 and 2004, was nominated for Most Outstanding Actor for his role in Stingers.
In 2011, Sweet was initiated into the Australian Film Walk of Fame on 13 March, as part of the closing night celebrations of the Australian Film Festival.
2004 saw Sweet become the public face of "The Performance Pack Initiative", an information campaign for men with impotence problems from Bayer, GlaxoSmithKline and Impotence Australia. Bans against direct advertising prescription medicines meant that Sweet was unable to directly mention the drug, Levitra, but this code was broken when Sweet mentioned Levitra on an Adelaide radio programme. The Australian Broadcasting Corporation's Media Watch reported that Sweet was being investigated by the Therapeutic Goods Administration for possible breach of advertising codes.
Sweet is divorced from his third wife, television presenter and former Commonwealth Games swimmer, Johanna Griggs. They have two sons, Jesse James and Joe Buster. Sweet has two children from his second marriage to Jill Miller, Frank and Sophie. Following in his father's footsteps, Frank Sweet is also an actor. Sweet's first marriage was to actress Lenore Smith.
Gary Sweet is an ambassador for the Glenelg Tigers, an Australian rules football club who play in the South Australian National Football League. He is also a supporter of the Manly Sea Eagles rugby league club who play in the National Rugby League.