While the exact location isn't specified, we know it's London because:
* The description matches London: Hagrid flies over "a huge, dark city," which is a fitting description for London with its size and historical architecture.
* Later events take place in London: The rest of the story takes place in London, including Harry's life at Privet Drive and the events at Hogwarts.
Therefore, it's safe to assume that Hagrid flew over London on his way to Privet Drive with baby Harry.