She was born Rose Marie Hinton into a farming family in Oneida, Arkansas, and later married James McCoy. She moved to New York City in 1942 to pursue a singing career. After starting her career singing in bars in New Jersey, her vocal talent got her bookings at famous venues such as Harlem’s Baby Grand, Detroit’s Flame Show Bar, Cincinnati's Sportsmen's Club, and Toronto's Basin Street.
In 1952, Rose Marie McCoy wrote and recorded two songs for the newly formed rhythm and blues label Wheeler Records, “Cheating Blues” and “Georgie Boy Blues”. After publishers heard these songs they sought her out, and she started working in the Brill Building. One of the first songs she was asked to write was a half-spoken, half-sung song, “Gabbin’ Blues”, co-written with Leroy Kirkland, and sung by Big Maybelle with the spoken part provided by McCoy herself. “Gabbin’ Blues”, which reached #3 on the Billboard R&B chart, was the first big hit for Big Maybelle and the songwriter’s first hit.
McCoy wrote other songs for Big Maybelle, and other popular R&B artists including Louis Jordan (“If I Had Any Sense I’d Go Back Home” and “House Party”) and co-wrote, with Fred Mendelsohn, Nappy Brown's 1955 single "Don't Be Angry" (also recorded for the pop market by the Crew-Cuts).
In 1954, Rose Marie McCoy teamed with songwriter Charles Singleton. They soon scored their first hit, “It Hurts Me to My Heart”, recorded in 1954 by Faye Adams. Their collaboration lasted about eight years and, individually and together, they penned many hits for the top artists of the time, including Elvis Presley’s "I Beg Of You", The Eagles' “Trying to Get to You" (later recorded in Presley's Sun Sessions), Ruth Brown’s “Mambo Baby”, and Nappy Brown’s “Little by Little”. Singleton & McCoy tunes were also recorded by Nat King Cole ("If I May", "My Personal Possession"), Little Willie John ("Letter from My Darling"), Eartha Kitt, Aretha Franklin, Eddy Arnold, The Five Willows, Big Joe Turner, The Du Droppers, Little Esther, The Clovers, and many other top artists of the time.
After the Singleton and McCoy team split up, Rose Marie McCoy continued to write songs on her own and collaborated with other writers. Noted for her independent stance, McCoy turned down several opportunities to join major record labels such as Motown, Stax and Atlantic. Her most successful song of the 1960s was “It's Gonna Work Out Fine", co-written with Sylvia McKinney, which became Ike and Tina Turner’s first top 20 pop single in 1961 and their first Grammy nomination. She also collaborated successfully with songwriter Helen Miller, writing "We'll Cry Together" for Maxine Brown, and "Got to See If I Can't Get Mommy (To Come Back Home)" for Jerry Butler.
Though she is most often associated with songs recorded by R&B artists of the 1950s and 60s, Rose Marie McCoy has written many jazz, pop, rock ‘n’ roll, country, and gospel songs. Jazz vocalist Jimmy Scott recorded nine of her tunes, and Sarah Vaughan recorded six of her songs, five of them on the singer’s 1974 album Send in the Clowns'.
Many other artists have recorded some of the over 800 songs she published, including but not limited to: Pearl Bailey, Maxine Brown, Shirley Caesar, Les Paul & Mary Ford, Patti Page, Debbie Reynolds, Lenny Welch, Dinah Washington, Barbara Lewis, Del Shannon, Joe Medlin, Freddie Scott, Billy & Lillie, Tommy Sands, Marie Cole, Shirley and Lee, Sammy Turner, Solomon Burke, Toni Arden, The Crew-Cuts, Ellerine Harding, Annie Laurie, Al Hibbler, Vera Longus, Jimmy Rushing, Eartha Kitt, Otis Williams, Bette Midler, Billy Eckstine, Ella Mae Morse, Chuck Jackson, Eddy Arnold, Dizzy Gillespie, Brook Benton, Buddy Ace, Varetta Dillard, Ivory Joe Hunter, Big Dee Irwin, Jane Froman, Shirley Ellis, Jimmy Rushing, Peggy Lee, Jean Wells, Jo Stafford, Georgia Gibbs, Joe Erskine, Bobby Vee, Wilbert Harrison, Linda Hopkins, The Platters, The Four Preps, Dakota Staton, The Harptones, Moms Mabley, Etta James, Gloria Lynne, Faith Hill, Linda Ronstadt and James Taylor.
McCoy was a resident of Teaneck, New Jersey from the 1950s. She was honored by Community Works NYC in their 2008 exhibition and concert series "Ladies Singing the Blues." McCoy received a five-minute standing ovation during the award ceremony at the Cathedral Church of St. John the Divine in New York City for her contribution to music. To the delight of the audience, "It's gonna work out fine" was played as she was escorted to the stage.
A biography, Thought We Were Writing the Blues: But They Called It Rock 'n' Roll, was written by Arlene Corsano and published in 2014.
Rose Marie McCoy died at her home in Champaign, Illinois, on 20 January 2015 at the age of 92.