Decide what kind of collection you want to build. These books have been around for a long time and are available in a variety of editions. They come as inexpensive, individual paperbacks and as deluxe box sets. Some include all of the original illustrations; others have been re-illustrated.
Decide if you want to buy in person or online. If you're familiar with the stories and know what you need (inexpensive paperbacks for an energetic toddler, for example), shopping online is probably fine.
Screen the books for trouble if your child is very sensitive - these are not as grim as Grimm by any means, but there are moments that might disturb children. In "The Tale of Benjamin Bunny" for example, Potter writes, "I cannot draw you a picture of Peter and Benjamin underneath the basket, because it was quite dark and because the smell of onions was fearful; it made Peter Rabbit and little Benjamin cry."
Screen the stories for gender stereotypes. Potter was an Edwardian - in these books, all the girl animals wear long dresses; most cook and keep house and are less adventurous than the boy animals.