Contrary to some outdated guides, the Yellow Flute is not on Route 12 in Pokémon HeartGold. You'll find it on Route 36, in the small patch of tall grass just before the bridge leading to Goldenrod City.
Equip the flute on a Pokémon to attract the roaming legendary Raikou to that area, making it easier to encounter and catch.