Kemp was born in Key West, Florida, the daughter of Nancy Jean (Haycock) and Joseph Clifton Kemp, a business executive and Navy Officer/Test Pilot who was stationed at the Pentagon.
She demonstrated talent as a child painter from the ages of 13 to 16, graduating from high school with a special award for Creative Achievement. At age 16, she traveled to the Rhode Island School of Design but was told to wait a year. She instead went to New York City and studied at The Art Students League and American Academy of Dramatic Arts.
Kemp was in the original cast of The Best Little Whorehouse in Texas, which began at The Actors Studio off-Broadway before it transferred to Broadway and became a long-running smash hit. Her mentor, Elia Kazan, took Tennessee Williams to see Kemp in this production and when Williams was looking for the actress to play Baby Doll in the world premiere of one of his last plays, Tiger Tail.
After the performance Williams gave her the part; an experience that proved to be most extraordinary, in that she worked closely with Tennessee Williams daily in developing the role. In 1978, she appeared on Broadway in a minor part in Once in a Lifetime.
In 1980, Kemp made her feature film debut in the horror film He Knows You're Alone, opposite Caitlin O'Heaney and Tom Hanks. Kemp received the GLAAD Award for her work on L.A. Law (1986). As a director, Kemp was responsible for many productions at The Actors Studio including The Glass Menagerie, The Beauty Queen of Leenane, and the world premier of Free Gift Inside (by Edward Allan Baker). Internationally Kemp directed The Stronger and Homesick at Strindbergs Original Intima Theater in Stockholm and Dreamstories at The Claude LeLouche Theatre Cine 13 in Paris and at La Spazia Teatro in Rome. Kemp became a Lifetime Member of The Actors Studio in 1975.
After having appeared on several television series, such as L.A. Law, in the mid-1990s, Kemp left Los Angeles and returned to New York City, where she worked odd jobs, including waiting tables at a restaurant. Shortly after, Kemp began teaching acting at the Strasberg Institute, and shortly after became a member of the Acting Faculty—and later, chair— of The Actors Studio Drama School at Pace University. After having left Los Angeles and returned to New York City in the mid-1990s, Kemp was working as a waitress in a restaurant.
At the Actors Studio Drama School, Kemp was a mentor to actor and student Bradley Cooper. Kemp worked across the country at The California Actors Theatre in San Francisco, Alliance Theatre in Atlanta, Folger Theatre in Washington, D.C., Center Stage in Baltimore and Walnut Street Theatre in Philadelphia.
Kemp was an Artistic Associate and tutor at 16th Street Actors Studio in Melbourne, Australia.
Kemp married actor Michael Margotta in New York City in 1984. The couple were divorced at an undetermined date.
Kemp died of cancer on September 1, 2017 in Venice, Los Angeles, aged 65. Several public tributes were made to Kemp from her former students, including Bradley Cooper, Hugh Jackman, and Lady Gaga.