He started his senior career for Næstved IF, and made his debut for the Danish national team in a July 1980 friendly match against the Soviet Union. He had a trial with Arsenal FC in 1978–79, scoring a goal in a reserve team game. Ajax initially tried to sign Olsen in 1980, but it was only in July 1981 that he moved to play professionally for Ajax in the Netherlands.
While playing for Ajax, Olsen became well known for his part in the infamous "passed penalty" routine he performed with Johan Cruyff. When Cruyff was awarded a penalty kick in a Dutch Eredivisie league match against Helmond Sport on 5 December 1982, with Ajax leading 1–0, Cruyff passed the ball sideways to Olsen, who returned it to Cruyff, with the Dutchman then slotting the ball past the bemused goalkeeper to make the score 2–0. It was allowed by the referee because the penalty is a direct free kick, so it can also be taken indirectly.
During his stay with Ajax, Olsen earned the nickname De Vlo (The Flea) due to his posture and his ability to twist, turn and jump to avoid tackles. He was also called "The Untouchable". The Ajax coach in the 1981–82 season, Kurt Linder, regarded him the most surprisingly talented player in the young Ajax squad, both technically and tactically phenomenal.
He won the 1981–82 Eredivisie championship in his first season at the club. The next season, 1982–83, Ajax successfully defended the Eredivisie title, and also won the Dutch Cup to complete The Double. Olsen scored twice in the qualification tournament for Euro 1984, including a last-minute equaliser in the 2–2 draw against England. He was selected to represent Denmark at the finals tournament, and played two games, including Denmark's semi-final against Spain. The game ended a draw, but even though Olsen scored in the penalty shootout, Denmark were eliminated when Preben Elkjær missed his shot.
Olsen moved from Ajax to England in July 1984, joining Manchester United for a fee of £350,000, signed by manager Ron Atkinson. He scored a total of 24 goals for the Reds in more than four years at Old Trafford, including a hat-trick against West Bromwich Albion in a 3–0 Football League First Division home win on 22 February 1986. in his five-year spell at the club.
He collected an FA Cup winner's medal in 1985, when United beat Everton 1–0 in the final at Wembley, having ousted veteran Dutch winger and Arnold Muhren on the left wing that season. He also helped them establish a runaway lead at the top of the First Division in the first half of the 1985–86 season, where they won their first 10 league games and remained top of the table until well into the new year before floundering and finishing fourth in the table.
Already an established member of the Danish national team, he was called up to the Danish squad for the 1986 World Cup, and scored two goals in the three preliminary group stage matches, but he will always be remembered for making a grave mistake in the round of 16 match against Spain. He gave Denmark the lead on a penalty kick, but just before half-time he collected a ball from goalkeeper Lars Høgh. When he tried to return the ball to Høgh, his pass ended up straight in front of Emilio Butragueño instead. The Spanish striker levelled the game, and eventually went on to score four goals in the game that Denmark lost 5-1. Consequently, the term en rigtig Jesper Olsen (a real Jesper Olsen) entered the Danish lexicon.
During October 1986, Olsen had a training ground bust-up with midfield colleague Remi Moses and by the end of the month he was put on the transfer list by manager Ron Atkinson. However, Atkinson was sacked as manager a week later due to United's dismal start to the season, and his successor Alex Ferguson took Olsen off the transfer list and he would stay at Old Trafford for two more years, even passing up the chance to sign John Barnes from Watford in the summer of 1987 due to his faith in Olsen.
Through his later years at Old Trafford, Olsen had a hard time adapting to the English game, and saw his footballing development stagnate. He was selected for the Denmark team at the Euro 1988, but did not play any games at the tournament.
Olsen's Manchester United career ended in the 1988–89 season, and he transferred to Bordeaux of France for £400,000 in November 1988. In his final full season at Old Trafford, United had finished runners-up to Liverpool in the league. His final United goal had come on 12 December 1987 in a 3–1 home win over Oxford United.
In 1990, he left Bordeaux to play as a wingback for Caen. After suffering a serious injury, he left Caen and retired in 1992, despite being offered the chance to return to English football with Blackburn Rovers and Nottingham Forest.
He has maintained a low profile since his retirement as a player, and now lives in Brighton, Victoria, Australia. In 2003, he founded and commenced operating the Fun Football Group in Australia.
He was admitted to hospital on 4 May 2006 after suffering a subarachnoid haemorrhage. He had just returned from jogging, when he first felt the effects of the haemorrhage, and said, "It was frightening to feel the loss of control".
After recovering from the haemorrhage, Olsen visited Australia coach Guus Hiddink at the Australian World Cup preparation camp in Melbourne. Olsen was acquainted with Hiddink through his friends Søren Lerby and Frank Arnesen. "I'm doing well, but it was a quite scary experience", Olsen told the Dutch newspaper De Telegraaf. The newspaper described Jesper Olsen as looking somewhat fatigued, but otherwise appearing as his usual self.
In 2011 the Fun Football Group became a part of the Football Star Academy, of which Olsen was appointed as Director of coaching. He oversees the Elite training program, and passes on his worldly knowledge and experiences to new budding football stars.
Football Star Academy was instrumental in George Lambadaridis being given a chance to participate in European Football in a move to Club Belgian Club Brugge K.V., and works with Clubs all over Europe such as Panathinaikos F.C. in Greece, and Ipswich Town F.C. in the UK to link Australian talent to international opportunities.
Olsen became one of the Assistant Managers of A-League club, Melbourne Heart.Ajax
Eredivisie (2): 1981-82, 1982-83
KNVB Cup: 1983
FA Cup: 1985