In Louis Sachar's Holes, Armpit (Theodore) and X-Ray (Rex) never assess a "caveman" for toughness. The premise is a misunderstanding of a key scene.
Late in the novel, Stanley Yelnats IV and Zero (Hector Zeroni) discover a buried chest—not a caveman. Inside: stocks, deeds, jewelry belonging to Stanley's great-grandfather (stolen by Kissin' Kate Barlow). Also in the vicinity: the remains of a Native American man (implied to be a ancestor of Zero's / Sam the onion man's lineage), killed by settlers.
They don't evaluate him as "tough." They name him "Stanley Yelnats" (honoring Stanley's lineage) and recognize him as a brave warrior for his people—a victim of injustice, linking the past's cruelty to the present's corruption (Camp Green Lake).
"Caveman" might conflate: (1) the boys' primitive living conditions (digging holes, no hygiene), (2) the ancient remains, (3) a misremembered nickname. But the text is clear: they find a chest and human remains, not a living "caveman."
The moment is about historical justice circling back—not toughness testing.