While I'm sprinting hard and doing lots of skids is tough on your stems BMXers use stems that function the same way, if a little more robust, and they have no problems. They slam on them waaaaay harder and death is not an issue. Basically, unless you somehow forget to fasten your stem up, you should be ok. Even if you don't do it up properly and your bars do slip I don't think death is going to ensue.
I get the fear, but it is 100% irrational.