Toh began his career as an academic. He was a Reader in Physiology at the University of Singapore from 1958 to 1964. He became the Vice-Chancellor of the University of Singapore while concurrently serving as the Minister for Science and Technology from 1968 to 1975.
Toh became politically active during his time as a university student in London, when he served as the Chairman of the Malayan Forum, an anti-colonial group for students from Malaya and Singapore (which included two future Prime ministers of Singapore and Malaysia, Lee Kuan Yew and Tun Abdul Razak) which met regularly for discussions and debates on the future of the Malayan region.
Toh was among the founder members of the People's Action Party and the party's chairman from its formation in 1954 to 1981, save for a brief period in 1957 when leftists, who dominated the common membership in 1957, elected leftist leaders and took over the party leadership. The founding members were restored when many of the leftist leaders were arrested by Lim Yew Hock in his anti-communist crackdown, allowing for the restoration of the original "basement group" of Toh, Lee Kuan Yew and Goh Keng Swee, et al to the Central Executive Committee (CEC). Following this, Toh implemented a cadres system to prevent from the newcomer "ordinary members", including leftist sympathisers, from having undue influence over the membership of the CEC.
Toh was elected as a PAP member for Rochor in the 1959 Singapore general election.
Toh was a key member of Lee Kuan Yew's faction in their fight against their rivals within the party.
Following the PAP's victory at the 1959 elections, the members of the party's Central Executive Committee voted to decide whether Lee, as the party's Secretary-General, or the party's Treasurer, Ong Eng Guan (who served as the Mayor on the City Council from 1957 to 1959), should take up the newly created post of Prime Minister. The vote was tied (6–6), and Toh, as the party's chairman, used his casting vote in favour of Lee.
Toh was a tenacious fighter in the battle against the Barisan Sosialis party, a splinter group from the PAP. He managed to defeat Barisan chairman Lee Siew Choh by a mere 89 votes in the 1963 Singapore general election, his narrowest electoral victory.
Toh held several Cabinet portfolios in pre- and post-independence Singapore, including Deputy Prime Minister (1959–68), Minister for Science and Technology (1968–75) and Minister for Health (1975–81). He served as the University of Singapore's Vice-Chancellor from 1968 to 1975. His stint at the university drew mixed reactions from staff and students. While his role to reorient the university's focus to suit the fledging nation's needs was applauded, he was perceived as authoritarian, when he clamped down on student demonstrations and political activities.
Toh stepped down from the Cabinet and party chairmanship in 1981. He served a further two parliamentary terms as a vocal backbencher, during which time he criticised his own party on a regular basis. He retired from Parliament at the 1988 general election.
Toh was generally considered one of the founding fathers of modern Singapore that came along with Lee Kuan Yew, Goh Keng Swee and S. Rajaratnam, for helping to lead Singapore during the nation's formative years.
Majulah Singapura was chosen by Toh as the national anthem of Singapore. In 1959, he headed the team that designed the Singapore coat of arms and flag.
Toh completed his secondary education at St George's Institution in Taiping, Perak, and at the Anglo-Chinese School in Ipoh. He obtained a diploma in science from Raffles College in Singapore in 1946. He furthered his studies at University of London and in 1953 he received a PhD in Physiology from the National Institute for Medical Research.
Toh spent his last years away from the public eye. The Straits Times featured Toh twice in 2005 and 2006 respectively, once on 2 May 2005 to pay his last respects to former president Wee Kim Wee. He was seen being assisted by two men and a walking stick as he walked. In February 2006, Toh was featured again in The Straits Times paying his last respects to the late former Deputy Prime Minister S Rajaratnam at his home in Chancery Lane.
Toh was briefly back in the spotlight in 1991 when he was mistaken for the culprit in a "hit and run" motor accident by The New Paper, who published the story on the front page. The culprit was in fact another individual who was also named Toh Chin Chye. Toh was subsequently awarded damages in a libel suit against the paper.
Toh died in his sleep on 3 February 2012 at 9:30 am (GMT+8). He was 90 years of age. He is survived by his son-in-law and four grandchildren aged 4 to 15. He was given a private funeral according to his wishes on 7 February 2012 at the Mandai Crematorium. As a mark of respect for his contributions to Singapore, his coffin was draped in the national flag and borne on a ceremonial gun carriage to the crematorium. The Singaporean flag was flown at half-mast on the day of his funeral.