The difference comes down to one fundamental force: friction.
Pushing an empty cardboard box vs. the same box filled with books. The heavier load presses the surfaces together more tightly, increasing the interlocking of microscopic irregularities—resistance rises linearly with weight.
More weight = more normal force = more friction = more applied force needed to overcome static friction and initiate motion.
The coefficient of friction (mat material vs. floor material) stays constant—only the normal force changes. That's why the person's presence makes all the difference.