As Balochistan's longest-serving governor, Rahimuddin ended the 1973 operation in Balochistan, declaring a general amnesty and military withdrawal in 1978. His tenure saw widespread economic development and the halting of the Baloch insurgency, but was controversial for suppressing the Afghan mujahideen entering the province during the Soviet war in Afghanistan.
Rahimuddin Khan was born in Qaimganj, United Provinces, India, in the majority Afridi Pashtun community that migrated from Khyber Pakhtunkhwa. He was the nephew of educationist Zakir Hussain, later the President of India, and the son-in-law of educationist Mahmud Hussain. He attended Jamia Millia Islamia University in Delhi, founded by Zakir Hussain. He opted for Pakistan during independence in 1947, enrolled as Gentleman Cadet-1 of the Pakistan Military Academy. As captain, he helped enforce martial law in Lahore during the 1953 Lahore riots.
He attended Command and General Staff College at Fort Leavenworth, Kansas and Command and Staff College in Quetta in 1965, and was appointed sub-martial law administrator of Hyderabad in 1969. He served as inaugural commander of 111 Brigade in Rawalpindi in 1970. In February 1971, General Yahya Khan assigned Rahimuddin to preside over the military tribunal of Sheikh Mujibur Rahman in Faisalabad. According to Shuja Nawaz, Rahimuddin was reportedly uncomfortable conducting the trial. 111 Brigade was led by Rahimuddin, and later Naseerullah Babar at the trial's verdict, during the Indo-Pakistani war of 1971. After becoming Prime Minister, Zulfikar Ali Bhutto rescinded the verdict, which was never made public, and released Mujib.
Rahimuddin served as Chief Instructor at the Armed Forces War College at the then National Defence College, Rawalpindi until 1975. Bhutto requested Rahimuddin to head the new Atomic Energy Commission and nuclear programme, but was declined. As Lieutenant-General, he became Commander II Corps in Multan in 1976. He was made Chairman Joint Chiefs of Staff Committee by General Zia-ul-Haq in 1984. He retired in 1987.
A military operation against separatists was commenced in Balochistan by Prime Minister Bhutto led by army chief Tikka Khan in 1973, claiming thousands of lives. Rahimuddin was appointed Governor of Balochistan on September 16, 1978. He declared an end to the operation, and announced a general amnesty for fighters willing to give up arms. Army withdrawal was completed by 1979. The Baloch separatist movement came to a standstill. Under Rahimuddin, Foreign Policy Centre held that "the province's tribal sardars were taken out of the pale of politics for the first time." He was known for clean reputation during corrupt regimes.
Rahimuddin opened the Sui gas field to provide gas directly to Quetta and other Baloch towns for the first time. Electricity expansion from Quetta to Loralai converted vast areas with sub-soil water into fertile ones. He also consolidated the then-contentious integration of Gwadar into Balochistan, notified as a district in 1977. Despite opposition from finance minister Ghulam Ishaq Khan, Rahimuddin heavily promoted large-scale manufacturing and investment in infrastructure, leading to provincial GDP growth rising to the highest in Balochistan's history. Addressing the province's literacy rate, the lowest in the country, he administered the freeing up of resources towards education, created girls' incentive programs, and had several girls' schools built in Dera Bugti District. He also oversaw the construction of nuclear test sites in Chaghai where tests were conducted in 1998.
During the Soviet war in Afghanistan, the Zia regime began aiding the anti-communist Afghan mujahideen. Millions of Afghan refugees, believed to be the largest refugee population in the world, crossed over the border into Balochistan and Khyber-Pakhtunkhwa. Under Zia and General Fazle Haq in KP, heroin and weaponry freely entered with the mujahideen. In Balochistan however, Rahimuddin Khan detained the mujahideen in barbed wire military camps and seized their arms. Several fighters were allegedly transported back into Afghanistan by force, criticized by Pakistani human rights agencies. He also restricted refugees to civilian encampments during the war. Pakistan's Balochistan policy became highly unpopular in the eyes of Afghans, but drugs and weapons remained low in the province, becoming widespread in Khyber-Pakhtunkhwa.
In March 1981, the Al-Zulfikar terrorist organization led by Murtaza Bhutto hijacked a Pakistan International Airlines airplane from Karachi to Kabul. The hijackers threatened to murder hostages if state authorities did not release specific political prisoners. Upon the authorities' refusal, Al-Zulfikar shot and killed passenger Captain Tariq Rahim, mistakenly believed to be the son of General Rahimuddin Khan. The decision to kill Rahim was taken after Murtaza Bhutto consulted KHAD chief Mohammad Najibullah. Tariq Rahim had actually been a former aide-de-camp to the elder Bhutto. The episode was ended when Zia-ul-Haq released the prisoners.
Zia dismissed his own government in May 1988. Rahimuddin became civilian Governor of Sindh, and governor's rule was imposed citing emergency. Claiming corruption, Rahimuddin began dismissing large numbers of police and civil servants, including Z.A. Nizami from the Karachi Development Authority. Rahimuddin also launched a brutal police crackdown on land mafia, one of the widest ever in Karachi, criticized by both PPP and the Zia regime for its heavyhanded tactics. It was stopped by the government immediately after he resigned. He moved to create separate police forces for the city and the rural areas, but this was also resisted after his resignation for fears of complicating the Sindhi-Muhajir relationship. Special riot control officers were trained to cope with ethnic riots, and river and forest police were also set up to battle dacoity. Ghulam Ishaq Khan became acting President after Zia's death in an aircrash on 17 August, and reintroduced the Chief Minister of Sindh office. Rahimuddin resigned in response, some say as this was attempt to limit his gubernatorial powers.
Post-retirement, he projected his chiefs of staff Asif Nawaz and Waheed Kakar for army chief.