McKern was born in Sydney, New South Wales, the son of Vera (née Martin) and Norman Walton McKern. He attended Sydney Technical High School. After an accident at the age of 15, he lost his left eye. He first worked as an engineering apprentice, then as an artist, followed by service as a sapper with the Australian Army's Royal Australian Engineers during World War II. In 1944, in Sydney, he performed in his first stage role.
Having fallen in love with Australian actress Jane Holland, McKern moved to the United Kingdom to be with her; they married in 1946. He soon became a regular performer at London's Old Vic theatre and the Shakespeare Memorial Theatre (now the Royal Shakespeare Theatre) in Stratford-upon-Avon, despite the difficulties posed by his glass eye and Australian accent.
McKern's most notable Shakespearean role was as Iago in Othello, in 1952. In 1955 he appeared in "The Burnt Flower Bed" by Ugo Betti directed by Peter Hall at the Arts Theatre Club in London. He played Big Daddy in Peter Hall's production of Cat on a Hot Tin Roof at the Comedy Theatre in 1958 and went on to play the German ambassador in another Peter Hall production, Brouhaha starring Peter Sellers at the Aldwych Theatre. He originated the role of Common Man in Robert Bolt's A Man for All Seasons in the West End in 1960, but for the show's Broadway production appeared as Thomas Cromwell, 1st Earl of Essex, a role he would reprise for the 1966 film version. He also portrayed Subtle in Ben Jonson's The Alchemist in 1962. In 1965, he played the eponymous villain in Bolt's The Thwarting of Baron Bolligrew, and Disson in Harold Pinter's Tea Party.
He appeared at the Royal Exchange, Manchester in Uncle Vanya in 1977 and in Crime and Punishment in 1978.
McKern's film debut was in Murder in the Cathedral (1952). His other more notable film appearances included the science-fiction classics X the Unknown (1956), The Day the Earth Caught Fire (1961), the World War I drama King and Country (1964), Help! (1965), the Academy Award-winning adaptation of A Man for All Seasons (1966), The Shoes of the Fisherman (1968), Ryan's Daughter (1970), Massacre in Rome (1973), The Adventure of Sherlock Holmes' Smarter Brother (1975), The Omen (1976), The Blue Lagoon (1980), The French Lieutenant's Woman (1981) and Ladyhawke (1985). He was presented with the Australian Film Institute Award for Best Actor in a Leading Role for Travelling North (1987). In Monsignor Quixote (1985), he co-starred as Sancho Zancas opposite Alec Guinness as Father Quixote.
One of McKern's earliest television roles was in the 1950s black-and-white series The Adventures of Robin Hood (as Sir Roger DeLisle, usurper of the Locksley manor and lands, and Herbert of Doncaster, a corrupt moneylender). During the 1960s, he was one of several Number Twos in the groundbreaking and critically acclaimed TV series The Prisoner. Along with Colin Gordon, McKern was one of only two actors to play Number Two more than once. He first played the character in the episodes "The Chimes of Big Ben" and "Once Upon a Time", and later reprised the role in the final episode, "Fall Out". The filming of "Once Upon a Time" proved to be a particularly intense experience for McKern; according to one biographer, the stress caused him to suffer either a nervous breakdown or a heart attack (accounts differ), forcing production to stop for a time. In 1976 McKern narrated and presented The Battle of the Somme, a British Broadcasting Corporation documentary produced to mark the 60th anniversary of World War I battle. He played the Earl of Gloucester in the Granada Television production of King Lear (1983) with Sir Laurence Olivier. In 1983 also starring in most episodes of the mini-series 'Reilly, Ace of Spies' as 'Zaharov', director of Vickers with Sam Neill.
In 1975, McKern made his first appearance in the role that would make him a household name as an actor, Horace Rumpole, whom he played in Rumpole of the Bailey, originally an episode of the BBC's Play for Today. A series of the same name, comprising 44 episodes, was produced for ITV between 1978 and 1992. According to Mortimer, "he not only played the character Rumpole—he added to it, brightened it and brought it fully to life."
Although he enjoyed the role, McKern expressed doubts about its popularity and the extent to which his life was becoming intertwined with Rumpole's. "McKern was often unhappy, decrying his television fame as an "insatiable monster". He stressed that his Peer Gynt was a greater performance and lamented: "If I get an obit in any paper, they will say, '.. of course, known to millions as Rumpole.'" In the later series, his daughter Abigail McKern joined the cast as Liz Probert.
In 1987, investment firm Smith Barney selected McKern to succeed John Houseman as its spokesman. At the same time, Smith Barney's corporate advertising department decided to change the format of its television advertisements, the first of which featuring McKern airing in September 1987. McKern's advertisements were not as popular with the viewing public as Houseman's, and in 1989, Smith Barney switched to a campaign featuring the voice of American actor George C. Scott.
In the 1990s, McKern appeared in a series of advertisements for Lloyds Bank, widely shown on British television, in which he portrayed a character reminiscent of Rumpole.
McKern wrote one radio play, which became the film Chain of Events (1958). He also provided the voice of Captain Haddock in the 1992 and 1993 BBC Radio adaptation of Hergé's The Adventures of Tintin.
In 1983, McKern was appointed an Officer of the Order of Australia for his services to the performing arts.
McKern frequently travelled between England and Australia both to visit family and friends, and to appear in various films and plays. As he was frightened of flying, he booked tickets to travel on cargo ships. This gave him time and peace to read scripts and contracts, with the added benefit of feeling he was on holidays.
McKern informed his daughter Abigail that he suffered from stage fright, which became harder to control with age. He also worried that his stout frame would not appeal to audiences.
In 1997 he appeared in a party political broadcast for the UK Independence Party.
Suffering from diabetes and other health problems, he moved to a nursing home near Bath in Somerset in 2002. He died a few weeks later at the age of 82. His body was cremated at Haycombe Cemetery in Bath.
McKern and his wife Jane Holland had two daughters.