Determine if the book in question is, in fact, rare. Enter the author and title information in a reputable search engine like Alibris or AbeBooks.com, both of which have a rare books section. Note if the book is a first edition or if it is signed. If the website generates a list of copies that are available, their prices may indicate the range of sale prices.
Note how the description of the listings on the search engines affect their prices. Be honest about the condition of the book in question when estimating how much it could be worth based upon an initial search. Every detail factors into the book's condition, including whether it has its dust jacket, if it is a hardcover or whether its spine has been creased from reading if it is a paperback. Signs of shelf life, like bumped page edges, also matter. An author signature will increase a rare book's value, but any other writing or underlining will lower its value, sometimes drastically.
Contact and visit a local used bookseller to verify the condition quality of the book in question if an initial search has produced particularly high prices. In order for a rare book to be worth its listed high price, it has to be in impeccable condition. Someone working in the bookselling industry should examine the book in question.
Weigh your selling options because they will determine the worth of the book. One option is to create an account and post a listing on websites like Alibris, AbeBooks.com or Amazon.com. To be competitive, list the book just under the lowest price for its condition level. It may sell faster if it has the lowest price. Understand that these sites will charge a commission and payment service fee once the book sells. Another option is to sell the book with the consulted bookseller. Understand that the bookseller will not buy the book at its full value. Not only does he need to make a profit to support the business, he also has to weigh how long it will take to sell, or how much of an investment he might have to make. Depending upon these factors, the bookseller may not choose to purchase the book despite its potential value.