Hire an editor with an understanding of your subject to not only check spelling and grammar but to also check your facts.
If you know the particular math book you want your article to appear in, find out when the next updated edition of the book will be published. Do this by calling the publisher. Publisher's contact information can be found in "Books in Print" at your library.
Contact the publisher of the book and determine the submission requirements. If the book is a new edition rather than an updated edition, ask when the deadline for submissions is.
Submit your writing with a cover letter that lists your qualifications and the audience for your article. Mention other places you've been published and your education.
Use an agent, if needed, who is experienced in this area of publishing. Weigh the cost of hiring an agent against the benefit of having your article published.
Check academic and university presses---not just trade publishers. Academic presses are usually looking for scholarly, technical material. There are more opportunities for publishing with these companies. But your article will probably reach a smaller audience.
Consider having your article published in a scientific journal. It's not a book, but it is a step toward being published.