No. African drumming is a cultural tradition open to anyone, regardless of religious affiliation. Playing the djembe, dunun, talking drum, or any African percussion instrument does not require or imply adherence to any faith.
Some African-derived religions—Candomblé (Brazil), Santería (Cuba) —incorporate specific drum patterns, rhythms, and ceremonies as part of worship. Participating in those ceremonies involves religious context, but learning or performing the drumming itself does not.
The drums are a language; you can learn the vocabulary without adopting the theology.