Yes, these four words form a perfect rhyme group. All are single-syllable words ending in the /aʊ/ diphthong (the "ow" sound as in "cow"), making them phonetically identical in their final syllable.
This rhyme family demonstrates how English spelling can obscure sound relationships. Despite different vowel spellings—ow in "how," ow in "cow," ow in "bow," and ow in "row"—they share the exact same vowel sound. Note that "bow" and "row" each have alternate pronunciations (as in "bow and arrow" or "row a boat") that don't rhyme with this group, but their primary pronunciations align perfectly.
Such rhyme clusters are valuable for poets, lyricists, and language learners exploring English phonology. They illustrate how orthography and phonology diverge, creating both challenges and creative opportunities in verse.