Khandker was born in a respected Muslim family of Pabna in 1930. His Father was Late Khandker Abdul Latif and his mother was Late Arefa Khatun. He completed his Matriculation in 1947 and ISC in 1949. He Completed his Graduation from PAF College in September 1952 and PSA in 1965 from Pakistan Air Force Staff College. He started his service career as GD Pilot in 1951. He served in Fighter Squadron till 1955 and became Flying Instructor. He was at PAF Academy till 1957 as flying instructor. He served as Flight Commander at Flying Instructors' School till 1958. Later he became Flight Commander at Jet Fighter Conversion Squadron where he served till 1960. He was Squadron Commander at Pakistan Air Force Academy till 1961. Afterwards, he became Squadron Commander of Jet Fighter Conversion Squadron where he served till 1965. He served as Officer Commanding of Training wing at PAF Academy in 1966. He became PSA in 1965 from Pakistan Air Force Staff College. A K Khandker was President of PAF Planning Board from 1966 to 1969. Later, he was posted at Dhaka as Second in Command of PAF Base in 1969.
He was posted in the Pakistan Air force Base in Dhaka as Second -in -Command when the Liberation War began in 1971. He defected with 07 Bengali officers and some airmen from the Dhaka base of the Pakistan Air Force for Tripura on 12 May 1971 for joining the liberation war and reached Matinnagar, Tripura on 15 May 1971. The then Bangladesh Government of Mujibnagar appointed him as the Deputy Chief of Staff of the Bangladesh Armed Forces, and was given the responsibility of operation and also training of the freedom fighters. He closely interacted with the senior officials of the Indian Eastern Command located at Fort William, Kolkata about training strategies for the Freedom Fighters as well as for overall operation. He established the first ever Bangladesh Air Force in Dimapur, Nagaland during the war. The Bangladesh Air Force with its limited manpower and resources of only 09 officers, 57 airmen, and 03 aircraft carried out significant numbers of operation against the Pakistani invaders. He represented Bangladesh in the surrender ceremony of the Pakistani forces to the allied forces on 16 December 1971 at the Race Course ground. He received the gallantry award of Bir Uttom in 1972 for his luminous role in the Liberation War.
Bangabandhu Sheikh Mujibur Rahman, following the independence, appointed him as the Chief of Air Staff of the reconstituted Air Force. He left a lasting mark in developing the newly constituted Air Force. Within next two years, he assembled a fighter squadron, a helicopter squadron, and 2 radar units to its strength. He served the Bangladesh Air Force as its Chief during the period 1972–1975. Apart from serving the Bangladesh Air Force, he was the first Chairman of the national carrier Bangladesh Biman for the period 1972–1973. In the aftermath of the assassination of Sheikh Mujib in 1975, he was the first officer to step down from Government position as the Chief of the Bangladesh Air force as a mark of protest.
During the tenure of the immediate past Government of 2001–2006, Mr. Khandker is credited as the main architect of establishing the "Sector Commanders Forum" by organising Sector and Sub-Sector Commanders of the liberation war that waged strong movement against pro-Jamat and other alleged war criminals. Mr. Khandker, as the Chairman of the Forum assumed overall responsibility of guiding this movement. The "Sector Commanders Forum" constituted on 9 December 2006 launched courageous movement, conducted mass awareness program, organised meetings and processions across the country and formed human chains to raise public awareness and glorified our war of liberation that has been systematically undermined by the anti-liberation forces. People upraising as a sequel of this campaign helped in obtaining massive verdict in favour of the pro-liberation forces in the general election of 29 December 2008.
In a statement by the government, in recalling his remarkable contribution in directing the war of liberation as the Deputy Chief of Staff, in recognition of his role in reconstituting the Bangladesh Air Force to a professional and effective outfit of our defence system as its Chief following the independence, and for his dedicated service to the motherland in various important state positions and in appreciation for his arduous and relentless service to the Republic, the name of Air Vice Marshal (Rtd.) Abdul Karim Khandker is awarded the Independence Award for 2011.
He was appointed as the Bangladesh High Commissioner in Australia and served the mission during 1976–1982. Later on, he became the Bangladesh High Commissioner in India for the period 1982–1986. In 1986, he was appointed as Adviser to the President and thereafter he served the nation as the Planning Minister up to 1990. He was elected as a member of parliament in 1998 and 2009 from the Pabna–2 constituency (Sujanagar Upazila). In 2009, he had been inducted as a full cabinet minister and given the charge of the Minister, Ministry of Planning.
He has a wife, two sons and one daughter.
In 2014, Khandaker wrote his memoirs "1971 Bhetory Baire" (1971:Inside and Outside) and was praised by historian Sirajul Islam who said that "the book provided a balanced presentation of history and the outline of the war and the interest of different vested groups surrounding the war came up in it." The criticises the role of Awami League leadership during the Liberation saying that the political leadership had failed to play its due role.
However, it drew ire from the ruling Awami League government, for creating "distorting historic facts" the history of the Bangladesh Liberation War, as he wrote in his book that Bangabandhu Sheikh Mujibur Rahman ended his 7 March speech with "Joy Pakistan." An uproar in parliament ensued where Awami League lawmakers demanded that the book must be banned and its copies will be confiscated. They also demanded filing a sedition case against the writer, who is also a sector commander of the liberation war. Awami League lawmaker Sheikh Fazlul Karim, Selim said that the book "has distorted historic facts and violated the country's constitution by giving such misleading information on Bangabandhu.""I am in doubt whether he wrote the book by taking huge amount of money from any agency," he added. Planning Minsister Mohammad Nasim said "Bangabandhu Sheikh Mujibur Rahman awarded AK Khandaker with the Bir Uttam honours but he (Khandaker) insulted the father of the nation by extending his support to Khondaker Mostaq Ahmad. Prime Minister Sheikh Hasina made him a minister but now he wrote a book against Bangabandhu."
In response Khandaker resigned from the Sector Commanders Forum. He was then declared a persona non grata in the district of Gazipur and pro-government lawyers demanded withdrawal of all of his titles.