The song playing at the end of Elizabethtown when Orlando Bloom kisses Kirsten Dunst is "If I Ever Feel Better" by Phoenix.
It perfectly captures the bittersweet feeling of new beginnings and hope for a brighter future.