I wear ear buds while riding. I avoid the ones that are too successful at 'noise canceling' and I suggest the OP do the same. I don't think listening to music at a reasonable level makes me less aware of my surroundings in any meaningful sense. Cars go by on a regular basis, I'm certainly not relying on my hearing them to make me safe. I take steps to ensure I am visible, and I take a sensible lane position.

From experience on groups rides and various events, I've found that riders who are very concerned about traffic coming up behind them tend to ride in a much less safe fashion than others. Visibility, lane position, and predictability - these things keep you safe.