Born in Crosby, Lancashire, Wheeler played for Carlton, as an amateur, and Tranmere Rovers before he made the move to Burnden Park in 1950. He made his debut for Bolton Wanderers on 17 February 1951 in a home league match against Liverpool, it was a good day for Wanderers and Wheeler as they took the points in a 2–1 victory and after a decent debut he remained a virtual ever present for next year and a half. He got his first Wanderers goal exactly a month later on 17 March in a league fixture against Huddersfield Town at Burnden Park.
Wheeler tasted both joy and disappointment during the 1952–53 season: the joy came in the shape of Bolton reaching the FA Cup final at Wembley on 2 May 1953. They then went 3–1 up after 55 minutes, but with just 22 minutes remaining along came the disappointment, which was, obviously, much more hard to bear as Blackpool, led by Stan Mortensen (who scored a hat-trick) and Stanley Matthews (who had an absolute blinder) broke all Wanderers hearts by making the fight back to end all fight backs, winning, as they did, 4–3. The last 2 goals came in the 89th and 92nd minutes, with South African Bill Perry hitting the injury time winner after some fine work by the 'Wing Wizard,' Matthews; in fact, Matthews' performance was so good the final has been renamed the 'Matthews Final'.
Wheeler spent the best part of the 1953-54 season on the injury list as Bolton finished a creditable 5th. He did gain his one and only England cap in 1954, however, when Walter Winterbottom selected him to play in a British Home Championship match against Northern Ireland at Windsor Park, Belfast, goals from Johnny Haynes and Don Revie where enough the gain England a 2–0 win.
Wheeler won back his spot in the Bolton side where he remained for the next couple of seasons. He even received the honour of representing the Football League against the League of Ireland in Dublin during the 1954-55 season. In 1956 The side he made his Bolton debut against, Liverpool now in the 2nd Division, showed an interest in Johnny and Bolton agreed to sell him on 8 September with newly installed manager Phil Taylor paying £9,000 for experienced right-sided players signature.
Johnny joined his cousin, Ronnie Moran, and made his debut for his hometown club on 15 September 1956 in a league match at Anfield. Stoke City were the visitors and went back to the Victoria Ground with both the points after a 2–0 win, Wheeler's first goal came a month later on 13 October, it was another league fixture but this time it was at Ashton Gate, Wheeler's 7th-minute strike wasn't enough to prevent Bristol City winning 2–1.
Wheeler, noted for his ability to attack and defend settled straight into the Reds line-up using his vast experience to help steady a wobbling Liverpool ship. He impressed Taylor who made him skipper during the 1958-59 season. He joined the Reds as wing-half but his most memorable performance, a performance that put him in the record books, came as an inside-forward on 3 November 1956, Port Vale were the visitors for a league match and after keeping the Anfield club down to a single goal, from Billy Liddell, for 80 minutes, but then up popped Wheeler in the 81st with a goal, he then doubled his tally a minute later and rounded off one of the fastest ever hat-trick 3 minutes later in the 85th minute, a stunning achievement for a player not renowned for his goalscoring exploits, the game ended up as a 4–1 Liverpool win.
Johnny became a linchpin of the side over the next few campaigns appearing 43 times in '57/'58, 42 in '59/'60 and 31 in both '60/'61 and '61/'62, but he fell out of favour in the promotion winning season after Bill Shankly had brought in both Ron Yeats and Ian St John, in fact, Wheeler was only called upon once all season.
Johnny remained at Anfield during their first season back in the top flight but was never considered, the 33-year-old with 454 league appearances under his belt left Liverpool in the May 1963 when he joined New Brighton as an amateur, after his spell at New Brighton he finally retired.
Johnny married Freda and has three children: Gillian, Michael and Susan; His wife, Freda, died on 3 October 2007.