Buy used books. Probably the most important tip, this will save you a lot of money to start with. It will also make books available that are out of print or otherwise unavailable. Look in the local phone book for used book stores and get to know the staff. Once they get to know you they will keep an eye out for your favorites.
Shop at thrift shops. You'll be amazed at the variety of books you'll find that way, frequently for pennies on the dollar. Again, get to know the staff. You may get a chance to review new inventory before it gets to the sales counter.
Check out your local library. Not the stacks, but the sale counters. Most libraries have lots of donated books that are never loaned. They're used solely to raise money for the library.
Visit yard sales, garage sales, tag sales, whatever ways that your neighbors dispose of their unwanted but still good items. Lots of times, this includes books. It's also a great way to schmooze and get to know your neighbors or other buyers who can recommend new-to-you authors.
Do a web search for cheap books, discount books, used books, or whatever you can think of. You'll be amazed at the sources that come up. Most of these sites sell new and used books, and the quality of the used books can be very close to new for much less money.
Check out online auction sites. These days, most of them sell both used and new books, at excellent prices.