I also listen to music but use an external bluetooth speaker, which regularly gets drowned out by wind noise and traffic noise.
For me, wind noise is an issue at any and all times, even when there is hardly any traffic. And when there's little traffic, I think a car is more likely to sneak up on me if I happen to not be looking in my mirror as often as I should.