Terrell was born in Philadelphia, Pennsylvania, and began her career as a teenager, singing in various local groups. In 1961, she signed with Motown Records and released her debut solo single, "I Can't Believe You Love Me". The song was a minor hit, but it helped Terrell to gain recognition within the Motown organization.
In 1964, Terrell recorded the duet "Ain't No Mountain High Enough" with Marvin Gaye. The song became a major hit, reaching number one on the Billboard Hot 100 chart and winning two Grammy Awards. Terrell and Gaye continued to collaborate, and they released several more successful duets, including "Your Precious Love", "You're All I Need to Get By", and "If I Could Build My Whole World Around You".
In 1967, Terrell was diagnosed with brain cancer. She underwent surgery and treatment, but the cancer returned in 1969. Terrell died in Philadelphia on March 16, 1970, at the age of 24.