Potter was born in Leicester. Potter's family moved to Wales, and young Philip learned to speak fluent Welsh as a boy, while his father worked on rocket technology during the war. He sang as a boy soprano in the church choir. Potter's family moved to Farnham, Surrey, where he played principal roles with the Farnham Amateur Operatic Society, including Strephon in Iolanthe (1955) and Barinkay in The Gypsy Baron (1956).
He studied singing at the Guildhall School of Music and Drama, where he won prizes as a tenor and graduated with a teaching degree. Potter taught school for a while, and then debuted in the 1958 production of Where's Charley? at the Palace Theatre. In 1959, he appeared in Marigold, Flower Drum Song and Chu Chin Chow.
Potter joined the D'Oyly Carte Opera Company in 1961, appearing as Earl Tolloller in Iolanthe and Colonel Fairfax in The Yeomen of the Guard. Soon afterwards, he assumed the additional roles of Frederic in The Pirates of Penzance, Prince Hilarion in Princess Ida, Nanki-Poo in The Mikado, and occasionally Marco and later, Luiz, in The Gondoliers. When Thomas Round left the company in 1964, Potter also took over the role of the Defendant in Trial by Jury and the Duke of Dunstable in Patience. He also sang Richard Dauntless in Ruddigore, beginning when that opera was revived in 1967.
While he was with D'Oyly Carte, Potter also appeared every two weeks on the Welsh television show, Land of Song. He left the company in 1970 (the year after his co-star, soprano Valerie Masterson, left the company) but returned as a guest artist to play several roles in 1970–71. In 1975, during the D'Oyly Carte Opera Company's centennial season, Potter was invited to participate in the final performance of Trial by Jury, in which the company's regular chorus was augmented by fourteen former stars of the company.
In the 1970s, Potter and his wife operated a 900-year-old Devonshire ale house. He rejoined the D'Oyly Carte for the 1979 season, appearing only as Nanki-Poo in The Mikado. He then continued to perform occasionally, appearing, for example, as Lord Tolloller with the Gilbert & Sullivan Opera Company in 2001 at the International Gilbert and Sullivan Festival, where he also occasionally gave master classes or participated in coachings and lectures for many years.
Potter died in 2016 at the age of 80.
With the D'Oyly Carte Opera Company and Decca Records, he recorded the Duke of Dunstable (1961), Colonel Fairfax (1964), Hilarion (1964), Nanki-Poo in "A Gilbert & Sullivan Spectacular" (1965) (excerpts), and Frederic (1968), as well as "Songs and Snatches" (1970) (excerpts from several operas). Potter also participated in a 1965 BBC television broadcast of Patience as the Duke of Dunstable, and played Nanki-Poo in 1966 film version of The Mikado.