If you've never raced, you have no idea whether or not you have the fitness to "succeed". Before I started racing I was one of the fastest people on the group rides, and could outclimb nearly everyone I knew, but I still got blown out the back of my first crit within the first few laps. I was pulled as I about to be lapped 20 minutes in, and my average speed up to that point was nearly 26mph. Everything I thought I knew was wrong. Maybe you are strong enough to do well, but you really can't know that until you try. Until then, the rest of us will just roll our eyes while you come here and telling everyone how great you are with no experience...

Bringing this back to the OP, it only takes one or two super-freaks to really make the field suffer. It's kind of the luck of the draw. I've seen cat 5 races where there was nobody particularly strong and the field limped around in the low-20s for half an hour, but on the other hand I've also seen cat 5 races where a handful of really strong guys strung out the field in the high-20s leaving a string of shelled riders in their wake. A race like Sea Otter increases the likelyhood of these strong riders exponentially. But again, it's not the average "pace" that shells most riders, it's the max speed and the surges. Throw in a climb like at Laguna Seca and you better be a strong climber too. It's not a huge climb but it has a steep section and you have to do it over and over again. Anaerobic intervals to the max.