Find a long beard and wig in silver. Dumbledore's beard is long enough to tuck into his belt. Many costume shops have Dumbledore masks in their Harry Potter sections, complete with hair and beard.
Wear half-moon spectacles. He is commonly seen with these glasses at the end of his nose. Remember this when wearing your spectacles.
Sport a long purple cloak. You can see Dumbledore striding through the halls of Hogwarts his purple cloak billowing behind him.
Put on a tassel hat. One with secret hidden pockets inside will be much closer to Dumbledore's own hat.
Acquire a fake broken nose. Dumbledore's nose appears to be broken at least twice. This will give believability to your costume. If you purchase a mask, this feature will be included.
Carry yourself with pride and knowledge. According to his chocolate frog card, Dumbledore is the "greatest wizard of modern time." While dressing up carrying yourself accordingly will help you get into the role.