A better way of phrasing your question is, "Should I start off doing Cat 1-2 races for guys that have more than 20 years of racing experience or should I do Cat 5 races where everyone is like me?"
I'm a (weak) Cat 3 and I avoid most Masters races. They're too hard. 30+ is insane. 35+ is manageable for easy races. 40+ I've never entered. To get an idea of my wimp level (or HTFU level) I also don't do P123 races in general, which is about what a Masters race is.
One thing that illustrates this well is checking results for Masters and Cat 3s (or 1-2-3s) on the same day at various races last year. You'll see the same names up there in both races.
cdr