Greer was born in Fairview, Missouri, the son of Bernice Irene (née Dabbs), a speech teacher, and Randall Alexander Greer, a druggist. Greer moved to Anderson as an infant with his family. At the age of eight, he began acting in children's theater productions. He attended Drury University in Springfield, Missouri, where he was a member of Theta Kappa Nu.
Greer made his film debut as an extra in the 1938 film Jesse James, which was filmed mainly around Pineville, Missouri. He told the Neosho Daily News in 2002, "They were paying $5 a day – a day! – to local people for being extras. That was really good money in those days, more money than we had seen in a long time." Greer was recognizable to fans of Adventures of Superman, as he appeared in three episodes of that series, including the inaugural entry, "Superman on Earth" (1952), in which he was cast as the first person ever to be saved by Superman. He was the major guest star, as a man framed for murder in "Five Minutes to Doom" (1954), and as an eccentric millionaire in "The Superman Silver Mine" (1958).
Greer made hundreds of appearances in nearly 200 different television series, including the role of the marshal in the two-part "King of the Dakotas" (1955) and as Ray in "Paper Gunman" of the NBC western anthology series Frontier. In the 1956 movie Hot Rod Girl he played the auto repair shop owner Mr. Fry.
In 1957, he appeared in the episode "Revenge" of the syndicated crime drama Sheriff of Cochise and as Sanders in the episode "My Horse Ajax" of NBC's children's western series Fury, starring Peter Graves and Bobby Diamond. At that time, he starred in three episodes as Mr. Blandish in the syndicated romantic comedy series How to Marry a Millionaire (1957–1959) with Barbara Eden and Merry Anders. He guest-starred about this time on the syndicated adventure series Whirlybirds, starring Kenneth Tobey and Rescue 8, starring Jim Davis and Lang Jeffries. He joined David Janssen in an episode of Richard Diamond, Private Detective.
Greer was cast on the syndicated western series Pony Express, starring Grant Sullivan. Greer guest-starred as well on three CBS western series, Steve McQueen's Wanted: Dead or Alive, Trackdown starring Robert Culp, and Don Durant's Johnny Ringo. Thereafter, he appeared in the NBC modern western series Empire, and he also guest-starred on Jack Lord's drama series about the rodeo circuit, Stoney Burke.
Greer appeared in the 1957 episode "Ambush at Gila Gulch" of ABC's Tombstone Territory, the 1957 episode "Rebel Christmas" of the Tod Andrews syndicated series The Gray Ghost, and as Ed Grimes on the 1958 episode "312 Vertical" of Rod Cameron's syndicated series State Trooper. He appeared too in It! The Terror from Beyond Space (1958).
Other appearances in 1959 included the episode "Peligroso" on NBC's western series The Restless Gun and episodes of the syndicated Man Without a Gun. In addition, he had a recurring role as storekeeper Mr. Jonas and in one episode as Chester's uncle on Gunsmoke. He also acted with Keenan Wynn and Bob Mathias in the NBC adventure series The Troubleshooters. In 1960 he appeared in the episode "The Proud Man" in the role of Willie Medford on the syndicated western series Two Faces West and in the episode "Dark Fear" of CBS's anthology series The DuPont Show with June Allyson.
He was cast in The Twilight Zone in the 1962 episode titled "Hocus-Pocus and Frisby" and the 1963 episode "Valley of the Shadow", and in a 1963 segment of Jack Palance's ABC circus drama The Greatest Show on Earth. In 1967, Greer appeared in the series finale entitled "Elizabeth's Odyssey" of Barry Sullivan's NBC western series The Road West.
The 1960s brought Greer several more recurring roles in popular series, such as track coach Ossie Weiss in Hank and Sheriff Norris "Norrie" Coolidge in The Ghost & Mrs. Muir. Greer also made many appearances on the very popular series The Rifleman, starring Chuck Connors and Johnny Crawford, playing both good-guy and bad-guy characters.
In 1962, on the ABC/WB western series Lawman, in an episode titled "The Unmasked," starring John Russell and Peter Brown, Greer was cast in an entirely fictitious portrayal of Boston Corbett, the Union Army soldier who shot and mortally wounded John Wilkes Booth, the assassin of Abraham Lincoln. In that episode, Corbett is given the name "Joe Brockway" and depicted as a hotel owner in Laramie, Wyoming. In the storyline, two former Confederates from Georgia, played by Barry Atwater and Charles Maxwell, arrive in Laramie in search of Brockway, the key, they claim, to the settlement of an estate to which they are all a party. Jack Albertson is cast with Greer in the episode as a traveling elixir salesman named "Doc" Peters.
In 1963, Greer was cast as Jack Tabor in the Perry Mason episode "The Case of the Skeleton's Closet." He guest-starred in seven other Perry Mason episodes. He portrayed, for example, a drunkard in "The Case of the Left-Handed Liar"; a murderer in two other episodes, "The Case of the Prudent Prosecutor" and "The Case of the Lavender Lipstick"; and a murder victim in "The Case of The Fugitive Nurse." Greer can also be seen alongside other guest actors (Dewey Martin, Patricia Crowley, Ray Danton, Andrew Duggan, Walter Burke, David Carradine, and Dick Wessel) in a 1964 episode of Arrest and Trial titled "The Black Flower," portraying a store owner wounded in a robbery. That same year he performed in an episode of The Outer Limits, "The Children of Spider County," in which he is the protective father of a country girl who is enamored with the son of an extraterrestrial. Later, in 1967, Greer, then played an alien himself, posing as a Catholic priest in "The Experiment," an episode of The Invaders.
Greer had a prominent continuing role as Reverend Alden in the NBC series Little House on the Prairie from 1974 to 1983. Often cast as a minister, he performed the marriages of Rob and Laura Petrie on The Dick Van Dyke Show and Mike and Carol Brady on The Brady Bunch; and from 1992 to 1996 he tended to the spiritual needs of the townfolk in fictional Rome, Wisconsin, as Reverend Henry Novotny in Picket Fences. He also had a guest appearance on an episode of Scott Baio's Charles in Charge in the role of Buzz Powell.
In the May 9, 1991, episode of L.A. Law titled "On the Toad Again," he played a character addicted to a "high" produced by licking the skin secretions of psychoactive toads. In the 1997 film Con Air, Greer appeared as the old man discovered hiding under a pickup truck at "Lerner Field" by Nicolas Cage's character Cameron Poe.
Greer's last feature film was a prominent role as the 108-year-old version of the character played by Tom Hanks in 1999's The Green Mile, 61 years after Greer was an extra in the 1938 film Jesse James. Greer's last television performance was in a 2003 episode of Lizzie McGuire. Most of Greer's work was in supporting roles, but he told the Albany Times Union in 2000 that "every character actor, in their own little sphere, is the lead."
Greer, a resident of Pasadena, California, from 1943, died in 2007 at Pasadena's Huntington Hospital after a battle with renal failure and heart disease. His body was transported back to his native state of Missouri, where it was buried in Peace Valley Cemetery in the town of Anderson.