The lyrics you describe—"something about you baby drives me crazy, that pretty girl"—do not match the Smiths' song "Pretty Girls Make Graves." That track's lyrics focus on a darker narrative ("She looked so pretty / Those pretty girls make graves").
Your description more closely resembles "Pretty Girl" by Maggie Lindemann or "Crazy" by various artists. To identify the song accurately, consider:
If you can provide more lyrics, the genre, or the era, a precise match will be easier.