When referencing the title of another's book in your own work, use the same capitalization and punctuation the author does. If there is a subtitle on the book, separate the title from the subtitle with a colon.
If you are creating your own book, you have creative license to write your title as you desire. As a general rule, capitalize the first word and other long or important words. These are usually all words that are not conjunctions or short prepositions.
The most common styles for citing sources within research papers or other contexts are Chicago, APA and MLA. All of these styles agree that book titles should be underlined within the text of a work that is referencing them. However, it is also acceptable, and often more common, to italicize the title of a work rather than underline it.
Chicago, APA and MLA styles all have standards for bibliographies or works cited pages. For all, book titles should be italicized.