Abdul Qayyum Khan was from Chitral and had Kashmiri origin. His father, Khan Abdul Hakim, was originally from Jammu and Kashmir (Baramulla area), but worked as a Tehsildar in the North-West Frontier Province (N.W.F.P., now called Khyber Pakhtunkhwa in Pakistan in 2017) of British India.
Khan was educated at Aligarh Muslim University and the London School of Economics. He became a barrister.
Abdul Qayum Khan was one of the eminent lawyers of N.W.F.P. During his professional career he conducted some very important cases. He used to practice in criminal law. Mirza Shams ul Haq was his most trustworthy colleague, who remained always close to him during profession and politics. Abdul Qayum was also assisted in his chambers by Muhammad Nazirullah Khan advocate, who later served as a provincial secretary general and senior vice president of Pakistan Muslim League.
One of his brothers, Abdul Hamid Khan, was a prime minister of Azad Jammu and Kashmir, and another brother, Khan Abdul Rauf Khan, was a renowned lawyer.
Starting his political career in 1934 with the Indian National Congress, Khan quickly rose to serve as an elected member of the Central Legislative Assembly (1937–38) and the deputy leader of the Congress in the Assembly. At that time he admired Khan Abdul Ghaffar Khan. He authored a book, Gold and Guns, in which he praised Ghaffar Khan. He switched his loyalties to the Muslim League in 1945. He later claimed that Ghaffar Khan was plotting Jinah's assassination.
In the 1946 provincial elections, Khan campaigned for the All-India Muslim League along with Pir of Manki Sharif. However, the Muslim League won only 17 seats in comparison to the 30 seats of the Congress Party. The Congress Party formed the provincial government under the premiership of Khan Abdul Jabbar Khan (popularly known as "Dr. Khan Sahib").
Abdul Qayyum Khan was put in charge of destabilising the Congress government in the province through street agitations, ideological rhetoric and acquisition of sympathetic Muslim officers in the government. The presence of a Congress government at the extreme north-west of the Indian subcontinent was anomalous, and the province became a bone of contention between the Congress and the Muslim League as part of the Partition of India. Eventually, the British decided to hold a referendum to determine which dominion the province should go to. Abdul Ghaffar Khan demanded a separate nation of 'Pakhtunistan' comprising both the North-West Frontier Province and Pashtun parts of Afghanistan. When it was denied by the British Raj, he and his party boycotted the referendum held by the British government. The Muslim League won an easy victory for Pakistan (289,244 votes against 2,874 for India).
Within a week of the independence of Pakistan, the Congress government was dismissed under orders from Governor General Jinnah. Abdul Qayyum Khan was put in charge of a minority government on 23 August 1947. Khan navigated through the troubled waters ably, winning over enough Congress legislators to support his government.
However, Qayyum Khan faced internal dissensions. The Pir of Manki Sharif, who was a key figure in the campaign for referendum, was miffed that he was passed over. He objected to Khan holding both the premiership of the state and the presidency of the provincial Muslim League. The Pir gathered disgruntled legislators and intended to bring a vote of no-confidence against Khan. Khan diffused his efforts. Then the Pir formed a separate party under the banner of All Pakistan Awami Muslim League. An exasperated Qayyum Khan responded with "full fury and force". He externed the Pir of Manki Sharif from the NWFP and imprisoned nine other leaders. Despite the crackdown, the Awami Muslim League contested the provincial elections in 1951 to win 4 seats.
Qayyum Khan's administration was known for its development work in the province, including the construction of Peshawar University and the Warsak dam. He introduced compulsory free education up to middle school level in Frontier province, the first province of Pakistan to have this reform. He also made poor friendly amendments to the land revenue laws. He evoked opposition from a section of the feudal class due to his egalitarian policies. His political stand was opposition to the Khudai Khidmatgar movement of Ghaffar Khan. His alleged role in ordering the Babrra massacre is one which he faces much criticism. He led the Muslim League to a landslide victory in the 1951 elections, despite opposition from the Khudai Khidmatgar movement and opposition from federally backed fellow Muslim league opponents like Yusuf Khattak.
Qayyum Khan served as the Chief Minister till 23 April 1953.
He served as central minister for Industries, Food and Agriculture in 1953.
Arrested by the Ayub Khan regime, he was disqualified from politics and imprisoned for two years before finally being released.
Contesting the 1970 General Election in Pakistan from three seats as leader of the Pakistan Muslim League-Qayyum faction, he won two National Assembly of Pakistan seats, one provincial seat and, in 1973, entered into alliance with the Pakistan Peoples Party (PPP) after East Pakistan broke away in the Bangladesh Liberation War.
Appointed federal interior minister by Zulfiqar Bhutto, he served in that post till the 1977 elections, when his party suffered a near total rout. After Zia-ul-Haqs assumption of power, Qayyum Khan tried to unify all the disparate Muslim League factions. His efforts were inconclusive and he died on 22 October 1981.
He was always opposed by Khan Habibullah Khan; they were lifelong rivals since they were young class-mates at Islamia College, Peshawar.
Under the orders of Abdul Qayyum Khan the Babrra massacre occurred on 12 August 1948 in the Charsadda District of the North-West Frontier Province (now Khyber Pukhtunkhwa) of Pakistan, when innocent and unarmed workers of the Khudai Khidmatgar movement were brutally fired upon by the provincial government. Different sources stating different number of people (over 200 to 600) who were killed in this massacre. Among these victims there were also women who rushed to the scene to save their men. More than 1000 people were injured.In September 1948, then Chief Minister , Abdul Qayyum Khan gave a statement in the provincial assembly, "I had imposed section 144 at Babra. When the people did not disperse, then firing was opened on them. They were lucky that the police had finished ammunition; otherwise not a single soul would have been left alive". Khan Qayyum said hinting at the four members of the opposition in the provincial assembly. He said; "If they were killed, the government would not care about them.