He received his primary education at Tangkak Boys School (Sekolah Laki-Laki Tangkak) before entering Muar High School (Sekolah Tinggi Muar). Qualifying for a Colombo Plan scholarship, Abdul Ghani furthered his studies at La Trobe University in Melbourne, Australia. He graduated with a bachelor's degree in economics (Hons). Prior to returning to Malaysia, Abdul Ghani completed a master's degree in economics at the University of Queensland. Upon returning to his homeland, he joined the Economics Faculty of University Malaya (1974–1980) and within several years, was promoted to be the Dean of the School (1980–1984). In addition to that, he also served as Master of First College (Residential) for University of Malaya (1979 – 1984).
Abdul Ghani's political career began when he was first appointed to be a Senator (December 1984 - July 1986). This was followed by his appointment to the Federal Parliament in August 1986.
From May 1987 to November 1990, Abdul Ghani held the post of Deputy Minister for Energy, Telecommunication and Post. Subsequently, he was then given the portfolio of Deputy Minister of Finance from 1990 to 1993.
In December 1993, Abdul Ghani was promoted to become the Minister of Youth and Sports (1 December 1993 - March 1995). He immediately initiated a national youth program called 'Rakan Muda' (Young Friend), which helped mobilise Malaysian youths through healthy activities such as sports and cultural events.
In March 1995, Abdul Ghani was appointed to become the Menteri Besar of Johor and has held that position for the next 17 years.
During the Malaysian general election, 2013 however, he was defeated by Democratic Action Party's advisor Lim Kit Siang in Gelang Patah,Johor and was replaced by Mohamed Khaled Nordin as Menteri Besar for Johor.
The Sultan of Johor awarded Abdul Ghani the Order of the Crown of Johor; Seri Mahkota Johor (SMJ) medal and the Darjah Kebesaran Seri Paduka Mahkota Johor (SPMJ) in 1989, which brought with it the title of Dato'. Abdul Ghani was then conferred in 1996 the rank of "The Most Esteemed Family Order of Johor" (Darjah Kerabat Mahkota Johor Yang Amat Dihormati) (D.K. II)
On 9 April 2009, Sultan Iskandar bestowed upon Abdul Ghani the highest award in the State of Johor, DK (I). It is very rare for a commoner to receive such an award, which is normally given to royals.
Ghani did not agree with the Bangsa Malaysia concept and alleging that Bangsa Malaysia was a "nebulous concept" which overstepped the bounds of the Constitution. "Even if the term Bangsa Malaysia is to be used, it must only be applied in the context of all the peoples of Malaysia with the Malays as the pivotal race," he said.
Ghani insisted that those advocating Bangsa Malaysia were echoing Lee Kuan Yew's call for a Malaysian Malaysia made during the early 1960s when Singapore was a state in Malaysia, even though the campaign had been repudiated by the government.
After the 2006 UMNO Annual General Assembly, Ghani elaborated that his opposition to Bangsa Malaysia and Malaysian Malaysia was because:
It is about everything being equal and this does not capture the hearts of Malaysians. Therefore, regardless of whatever name is given, the concept is similar and this is against idea kenegaraan (the idea of nationhood) which we have inherited.
Ghani defined his idea kenegaraan as "the idea that has united the Malays, and also the same idea that has given privileges to other races to be citizens, live together and share power and prosperity."