The Order of the Phoenix novel does not mention a dedicated "Potions section" in the Hogwarts library. The library is described as vast with thousands of books on many subjects, but no specific Potions subsection is named.
Potions textbooks (e.g., Advanced Potion-Making) and reference works would logically be shelved there, likely under general magical subjects or a "Potions & Alchemy" classification, but J.K. Rowling never details the cataloging system.
In the films, the library set includes section signs, but "Potions" isn't explicitly shown.