My 2c:

Metal frames/parts usually show cracks on the outside, before completely failing.

With carbon fiber parts - it is not the case often enough. However - there are professionals who can properly inspect (and scan) carbon frames/parts to confirm it's all good. It costs, but can be done. With that precaution, perhaps even repeating the checkup in case of any accident, or larger scratches etc, suppose carbon parts are quite safe.

In my country, I don't know of anyone who offers the services of inspecting carbon parts, so for that reason (and the price of both frames, checkups and repairs), I still stay away from carbon components. I often ride without a helmet - but don't really like riding uninspected carbon forks, for example. Without an inspection - the risk is greater than any lower weight benefits, at least for the non-racing me. It seems that even high end manufacturers mess it up from time to time (educational Luescher Teknik videos).