Why H-G men are the antagonists in story harrison bergeron?

The story does not mention H-G Men as the antagonists or even as characters at all. The main and most frequent antagonists are the Handicappers from the United States Handicapper General's Office. They enforce a totalitarian dystopian vision of equality by ensuring nobody is allowed to stand out through intelligence, appearance, abilities, wealth, etc.

The Handicappers have a special role in upholding and enforcing an extreme form of egalitarianism, making them the primary conflict with the protagonist Harrison and ultimately leading to his rebellion/death at the end of the short story.