Decide if you will be using the clay for projects that you will want to keep or if you will just be playing with the clay. You will spend more on modeling clay that needs to be oven-dried or baked and painted. Modeling clay that is soft and easily reused is much cheaper.
Purchase clay that hardens if you plan to paint it. Look for clay that is oil-based. This kind of clay is usually not found in the toy stores, but is more likely to be found in art supply and craft stores. This kind of clay sometimes contains sulfur, so you may need gloves if you have sensitive skin.
Look for clay that is already colored, so you can create your pieces with the various colors and avoid the painting. To purchase clay that you use for artwork you plan to keep or give as a gift, craft and art stores are best option. These stores will also supply the accessories and tools you'll need to get the best work from your clay. Toy stores are the best source for modeling clay that your kids will use.
Buy good, tight storage containers, no matter what kind of clay you end up using. A tight seal on the container will allow you to keep and reuse modeling clay for a very long time.
Try various online vendors by buying small amounts before making a bulk purchase. Make sure the clay is of the quality you expect as there can be a wide range available.