So funny, I was thinking about this yesterday. I normally average only around 16 mph, but during the stretches where I'm at 24 +, I've noticed the wind noise as well. And it's night and day whether I put my head down or look up. It made me wonder about the pros who average in the 28 mph range... I wonder how they deal with the constant noise. They don't wear headphones or anything either (I wonder if it's a lot quieter inside the peloton)... My guess would be yes, but I wouldn't know as I've never ridden with anyone else.
Anyway, I use the Apple AirPod Pro (in my right ear only). They have active noise cancelling but I don't use that feature so I can hear what's around me. Regardless, when you're above 24 mph, headphones or not, you can't hear much.
I wouldn't use the earbuds if I rode in the city though (my normal route has a very few opportunities a car can turn in front of you).
Regarding long term hearing loss, I've spent years in semi-loud environments at work. It's not the loud spikes in noise, it's the constant drone over hours and hours a day that I've seen affect people (in the long term). I'm not sure a few hours a week on the bike would hurt you, but if you already have hearing loss, I can see a valid concern.