My 2c worth is that it certainly doesn't depend on what category they come from. I rode road a lot and must say I feel very safe around MOST of the high ranked road riders I've had the experience of riding with. Riding tight pacelines at high speeds on road sure as hell teaches you a thing or 2 about being a confident bike handler. I don't know about where everyone else rides, but where I ride, if you can't cut the mustard or your riding is a bit sketchy, then you are politely (and not so politely if the hint doesn't sink in) told to go sit on the back or leave all together. I know road and track riders with a lot of experience that I certainly don't like to rub shoulders with. I know a certain old masters rider that purposely rides sketchy to make riders feel they have to go high around him (I've trained with the guy and he can ride as straight as anyone).
I also know a former world champ masters woman who literally freaks out from time to time in bunch scenarios, but has held records and WCs in individual disciplines and national titles in bunch racing. She has the idea that if she stays in front she can't get taken out. Not a bad philosophy if you're strong enough to do it