Guess we shouldn't listen to music when we drive.... but
Cycling is already dangerous. I think the only reason to not use them is you add to the danger. Is it more dangerous than going 50mph+ down a hill? maybe, maybe not.
You might make it without any issues, that day.