Rousseau took piano lessons from his mother. In 1998, he acquired a copy of the Green Day album Dookie, and liked it a great deal. He asked his mother if he could play the guitar; she bought him an acoustic instrument. He learned to play it by ear, taking lessons briefly to be able to play Green Day songs. He attended the Friday night concerts at the Burlington YMCA, hearing many local bands, some of which were successful outside the area.
Rousseau had been a fan of the band Silverstein, and had been influenced by their debut album, When Broken Is Easily Fixed, one of the first albums with screamo characteristics that he had heard. In 2005, Rousseau attended the album launch show for Silverstein's second studio album, Discovering the Waterfront at the Burlington YMCA. Afterwards, he met the band's bass guitarist, Billy Hamilton, who invited him for a meal. Hamilton described Rousseau, then aged 14, as "one of the most hilarious, smart and talented kids I’ve ever met. He became sort of like a little brother to me." Over the following years, Hamilton would draw on his knowledge of music, setting tasks for Rousseau such as learning to play the second stanza of Billy Joel's Movin' Out (Anthony's Song).
In high school, Rousseau became part of a band called Japaned, and then joined a band known as Cain and Abel. In 2008, Rousseau joined a new band, I Am Committing a Sin, based out of Burlington, Ontario. All of I Am Committing a Sin's members had been part of Japaned, with the exception of Kelly Bilan, a member of Cain and Abel. After a year of limited success there, he expressed frustration, with the limitations of the local live music scene. I Am Committing a Sin, with an average age of 19, spoke out on political issues, and against organized religion, denying the concept of sin. The band did have its consolations, including the opportunity to tour with Silverstein.
After I Am Committing a Sin broke up in 2010, Rousseau recorded several songs for a project known as Burst & Bloom, which did not tour, though he posted the songs on Facebook. He sold most of his touring gear, including the van the band traveled in retaining only his guitar. He also helped Silverstein on their tours, serving as guitar tech, tour manager, and merch guy for the band.
In 2008, while dealing with a bad breakup, Rousseau was invited by a friend who was working for Silverstein on tour to come hangout on their bus to get his mind off his personal issues. After seeing what life was like being Silverstein's guitar tech, he asked the band if he could join them on tour as Neil Boshart's guitar tech, content to be paid in beer and, occasionally, pizza. The band agreed, and the then 19-year-old Paul Marc Rousseau began touring with the band as guitar tech.
On September 25, 2012, Silverstein announced on their Facebook that guitarist Neil Boshart was no longer a member of the band, after 11 years. They also announced Rousseau as Boshart's replacement, noting that as a member of the Silverstein "touring family", he was familiar with the band and its ways.
Silverstein's first album with Rousseau as a member was This Is How the Wind Shifts (2013), and band frontman Shane Told described Rousseau's presence as one of the keys to the band's artistic success. Among other contributions to the album, Rousseau wrote the first track, "Stand Amid the Roar", and the seventh track, "Arrivals".
In January 2014, while on tour with Silverstein in Canada, Rousseau fell ill and was taken to a hospital in Cornwall, Ontario, where he underwent emergency abdominal surgery. The band released a statement, announcing this and the cancellation of several shows. He recovered from the surgery, calling the slow recuperation difficult.
His second album as the lead guitarist of Silverstein, I Am Alive In Everything I Touch, is set to be released 19 May 2015 through Rise Records.
In 2013, Rousseau's side project, The Good Boys, was signed by Culvert Music. The Good Boys, a throwback to the days when bands wore collared shirts and neckties on stage, is described by their label as "more Ed Sullivan than MTV". All three of its members, including Rousseau, were members of "I Am Committing a Sin". It released its first single, "I'm Not Pretending" in September 2014, and plans to release its debut album in early 2015. One reviewer described The Good Boys' approach as to "showcase a straight-laced approach to their tracks along with releasing clean-cut, catchy pop-rock songs" The band performed the track on Breakfast Television on November 6, 2014.
"Plain Language" is an electronic duo teaming Rousseau and Koehler. It released two tracks, "Hoax" and "Softer Language", in mid-2014. Rousseau was a producer for the EP "Sun and Showers" by Toronto band Body Doubles.