You're close, but the distinction is more nuanced. Here's a clear breakdown:
This stat reduces damage from physical attacks—moves that involve direct contact like Tackle, Scratch, or Close Combat.
This stat reduces damage from special attacks—moves that use energy or elemental forces like Flamethrower, Thunderbolt, or Psychic.
In Generation I, there was only a single "Special" stat governing both special offense and defense. Starting in Generation II, this split into Special Attack (offense) and Special Defense (defense), adding strategic depth to team building.