Baby wipes are a useful and inexpensive option. Simply dab the ink stamp on the baby wipe or pick the stamp up and wipe it off thoroughly. Dry the stamp on a clean, absorbent paper towel. Make sure the stamp is completely dry before reusing it.
Use a stamp scrubber, a 4.1/2" x 7.1/2" plastic case with a scrubbing pad inside the case. Add a little water and dish detergent or stamp cleaner on half of the scrubbing pad and leave the other half dry. When you're finished using a particular stamp, drag the stamp across the wet half of the scrubbing pad. After the ink stamp is clean, press the stamp into the dry part of the scrubber. Again, make sure the stamp is completely dry before reusing it.
To make your own "stamp scrubber," dampen a sponge using water and a drop or two of dish detergent. Rub the ink stamp over the sponge until it is clean. Thoroughly dry the stamp using a paper towel.