In text (books, articles, essays): Album/CD titles are italicized.
Standing alone (e.g., on a cover, list, or marquee): They are not underlined or italicized.
This follows standard style guides (Chicago, MLA, AP) for titles of standalone works.