With good handling, fast reflexes, and a little bit of luck, I made it from 5 > 3 without any damage or injury from racing.
Cars and other road hazards are different story...
I'm a 4 now, made it through almost an entire collegiate season (still got conference champs coming up), and haven't wrecked in a race.
I've wrecked outside of races, I've wrecked in the parking lot before a race, but never in the race itself.
Cat 5 really isn't that bad. Many of my Cat 5 races had large fields and no crashes at all. Some collegiate weekends (some of the sketchiest riding there is, we're all fearless) we don't have any crashes at all, in any categories.