In the digital age, you can buy a loved one lots of music on the cheap. Digital music stores (such as iTunes, Amazon or Napster) frequently sell songs individually and by the album at a fraction of the price of physical formats such as CDs and records. Think about how much you want to spend, then buy the person a gift card to any popular online digital music seller.
Whether the person you're buying for is a musician or just a lover of music, magazine subscriptions are the gift that gives the whole year round. A number of magazines exist not just for specific instruments ("Guitar Player" or "Modern Drummer,") but also playing those instruments within a specific genre (such as "Just Jazz Guitar.") For the music lover, there are magazines which cover music generally ("Rolling Stone") as well as specific genres of music ("Spin" or "Metal Edge.") Think about what instruments and genres the person you're shopping for is interested in and get them a subscription to a relevant magazine.
Musicians can never get enough disposable equipment like strings, picks, cables and drum heads. Items such as these tend to be cheaper and can be bought in bulk. For the music lover, gifts like keychains, lighters and bottle openers with music-themed graphics on them can be found for under $15.
Many music lovers and musicians enjoy displaying their loyalties and influences with clothing. Not only do bands have licensed apparel, but you can also find t-shirts, hats and jackets that advertise a person's loyalty to a genre of music or instrument.
A book is an expensive gift for the music lover who wants to know more about their favorite artist or genre. Musicians may also be interested in histories and biographies; there are also scores of books published every year which contain instructions on different techniques or sheet music organized by artists, periods and genres. Many books about music can be bought for under $25.