He was born as one of the four sons of Sir Syed Wazir Hasan at Kalanpur, Jaunpur. The family belonged to the Shia sect of Islam, and were a land-owning family of the region. Zaheer's younger brother, Sajjad Zaheer, was a Communist leader, whose daughter Nadira Babbar is a theatre artist and the first wife of Raj Babbar, the film actor and Member of Parliament (MP).
He was married at a young age to Aliya, a distant cousin, in a match arranged by their parents. After Zaheer achieved prominence as a Muslim leader within the Congress Party, his wife was appointed chairperson of the Central Social Welfare Board. The couple had a son, Karim Zaheer, and a daughter, Sakina Hasan, who was a communist and a member of the Committee of the Status of Women appointed by Indira Gandhi under the chairmanship of Vina Mazumdar.
Zaheer began his career as a barrister in the Chiefs Court of Oudh (Awadh), based in Lucknow. He took interest in politics at an early age and joined the Indian National Congress. He selected Congress as they were leading India's struggle for independence from Britain.
In the 1930s and 1940s, actively encouraged by the policy of Divide and Rule, communalism was at its peak in India. Awadh, dominated by vastly rich Muslim landlords ruling an overwhelmingly Hindu peasantry, had become a hotbed of Muslim separatism, and indeed the main source of funding for the Muslim League, which wanted to partition India and create Pakistan.
Since many of the land-owners of the region were Muslims, much of provincial politics in Awadh during the days of the British Raj was dominated by them. Lucknow's was represented by the political rivalry between Chaudhry Khaliq-uz-zaman, a Sunni Muslim League leader, and Syed Ali Zaheer, a Shia Leader and a member of the Congress. Both fought elections for the Lucknow Municipal Board and the Congress won. Zaheer thus became Municipal Commissioner of Lucknow in the early 1940s.
Since he held an important position as a member of the Congress party, Zaheer refused to jump ship and join the Muslim League as many of his colleagues did. He was rewarded for this expedient loyalty in 1946 with the talks of independence beginning in right ernest.
When the provisional government of India was formed, in 1946, as a prelude to independence, Zaheer was appointed Law Minister. Later, he was appointed as India's ambassador to Iran, again by Jawaharlal Nehru. After his return, he resumed his legal practice.
Later, he became the Law Minister of Uttar Pradesh for two terms and also the Chairman of Uttar Pradesh Pay Commission.
Though he was a lifelong Congressman Zaheer briefly left the Congress during the general elections of 1977 and joined Babu Jagjivan Ram's Congress for Democracy (CFD).
He remained the president of the Shia Degree College Management Committee and was succeeded by his son Kazim Zaheer to that post.
Syed Ali Zaheer was buried in Daliganj Karbala, which is close to the Shia College there.