I ride with and without. When I ride where there are many people I don't wear them. When I ride at night at the park where I am the only one I wear them. On some other bike paths I wear them but put behind my ears so I can still hear the music but the sound of the world is primary. I think done responsibly (this the key here, right?) there is little danger. Heck we have some bikers riding on roads during busy times holding back traffic and that's generally accepted (It's legal, right?) but I'd say it is far more dangerous. I'm not saying one justifies the other but... anyways....
I ride my bike to enjoy my music. If I didn't have my music I'd ride far less.