The beginners in the Cross Crusade series race at the same time as the unicycles (who start a minute or two later). I've been passed by women, that's not a big deal, but there's nothing like the humiliation of being passed by a 13-year old on a unicycle who started a minute or more after you. I've seen it happen.
That said, we've got A, B, C and Beginner around here. I don't know if our beginners correspond to Cat 4 or Cat 5. I do know that they're people who haven't done a lot of races, and it's nice to have that group to get involved. It doesn't fix the problem of being spanked by super-fast racers who ought to be in a higher cat though, because every year there's somebody in their first year of CX racing who could do well in the B's. And if you're that slow guy in the second half of the beginner field, you need to learn to get used to being spanked by faster riders, because it's going to keep happening no matter what Cat you're in.
Seriously, as CX becomes more and more popular the number of people who are never going to win a race is going to go up proportionately. Most of us need to accept that we're not racing to win. And when you're racing to just do your personal best, it doesn't much matter whether you're 127th out of 155 or 34th out of 50. You get to know the guys you're racing against and that's your competition regardless of Cat.