Mahmud was born in Palembang, South Sumatra on 3 February 1930. His father, Masagus Mahmud, was a government official and his mother's name was Masayu Aisyah. Mahmud had 9 siblings—four older and five younger. As a child, Mahmud was interested in singing and dancing. He began formally studying music at the age of 11, after meeting then-14-year-old musician Ishak Mahmuddin. He later studied at the Jakarta Teacher's Training Institute.
Mahmud started his working life as a teacher in Tanjung Pinang, Riau. He later received a government scholarship to study English in Sydney, Australia. After finishing his studies in Australia, he moved to Jakarta and became a teacher at the School for Kindergarten Teachers. It was while teaching at the School for Kindergarten Teachers that Mahmud began to write songs, upon the request of his students. One of his first songs, "Cicak di Dinding" ("Lizard on the Wall"), was taught to kindergarten students throughout Jakarta by teachers that he had trained.
During his songwriting career, Mahmud wrote some 500 educational children's songs with simple lyrics and rhythms. Some of Mahmud's best known works include "Pelangi" ("Rainbow"), "Ambilkan Bulan" ("Take the Moon"), "Anak Gembala" ("Shepherd Boy"), "Bintang Kejora" ("The Morning Star"), "Mendaki Gunung", "Ade Irma Suryani", and "Amelia". Mahmud also hosted two children song's shows on TVRI, the Indonesian state-run television, "Lagu Pilihanku" (Songs of My Choice) from 1968 to 1988, and "Ayo Menyanyi" (Let’s Sing) from 1969 to 1988. Co-hosts include children's songwriters, Ibu Sud and Daljono.
Mahmud release his autobiography, A. T. Mahmud Meniti Pelangi: Sebuah Memoar (A. T. Mahmud Walking on the Rainbow: A Memoir, in 2003. It was published by Gramedia.
In 2009, Mahmud suffered a stroke. In early July 2010, he was hospitalized with a lung infection. After being released from the hospital, he died in his sleep at his home in Tebet, South Jakarta at around 1:00 pm (UTC+7) from the infection. He was buried in Menteng Pulo Cemetery, Jakarta on 7 July 2010, with well-wishers on Twitter and other social networking websites expressing their condolences.
Mahmud married Mulyani. Together they had a son and two daughters. At the time of his death, Mahmud had 7 grandchildren.
Mahmud believed that children should sing children's songs, and expressed disappointment at the trend for them to sing songs written for adults. He also believed that children's songs are not to entertain them, but to educate them and assist with their emotional development.
Mahmud has been described as a "legendary children's song writer" and "a true champion of children’s education". In 1999 he received the Artists Award from the Minister of Education and Culture. He was the recipient of the 2003 AMI Lifetime Achievement Award. That same year, the Indonesian government awarded him the Bintang Budaya Parama Dharma medal.
His work is credited with launching the career of child star Tasya. In 2000, his works (sung by Tasya) were compiled by Sony Wonder and released as Libur Tlah Tiba: Karya Abadi MT Mahmud (Holiday has Arrived: The Eternal Works of MT Mahmud), which went double platinum. The album was followed by two more: Gembira Berkumpul (Come Together in Excitement) and Ketupat Lebaran (A Rice Cake for Eid ul-Fitr).