I believe that the risk is quite low, unless you ride them after a crash and don't inspect them carefully for damage.

However, the weight savings is often marginal. Personally, I avoid CF in stems and bars.

But I suspect you will do fine if you want them. Remember to inspect very carefully after a crash, which probably involves unwrapping the bars etc.