Make a list of topic of ideas you want to find public domain books on. Having a list that narrows your search in the beginning will expedite the process, and make it much easier for you to uncover public domain books that interest you right away.
Head to your city library. The library is one of your best resources for locating public domain books, because you'll have an extensive catalog system at your fingertips that dates back for years. While at the library, you'll also have access to the librarians, who are experts in books and searching through records.
Register for a service that can assist you in tracking down public domain books. There are both online and offline services that, for a small fee, will compile a list of public domain books that meet your criteria. Most companies that offer this service can also assist you in making sure the books you are interested in are actually public domain, so you don't run into any trouble in the future.
Check out garage and estate sales. Many people hang onto books for generations and generations, only to have them sit and collect dust. These are often sold in garage and estate sales when they get around to doing some cleaning, and you can frequently pick them up for pennies on the dollar.
Visit an old and/or out-of-print bookstore. These are great places to find old public domain books for sale. They have shelves upon shelves of books that are legitimate public domain works, and generally have very reasonable prices.