It is wrong to assume everyone with earbuds/pods while riding is blaring music. Personally, I am a podcast junkie. Most of my rides are solo, and I frequently ride while listening to podcasts (often history-related podcasts). I keep the volume at a conversational level, and usually only use the right earbud. I wouldn't use them during a group ride out of courtesy, because someone might want to start up a conversation while riding. I can hear traffic just fine whether talking with a real person or listening to a podcast. I've never been on a group ride where there is a "no talking" rule.
Btw, if you can get past the slightly dorky look, these "cat ears" are helpful in reducing wind noise while riding. I use the slim version. I'm not sure how much they help with the speaker part, but they do definitely help you hear cars, calls, and podcasts while riding:
https://www.cat-ears.com/