I regularly race. 41 yrs old, 6'3", 255+ lbs.
I only do criteriums. Road races would be a waste of my time and money. I'm a cat 3 rider. I race Master's pretty regularly, and yes, I get to compete against former Olympians, ex-Domestic pros, and even the occasional ex-European pro. It is killer (understatement).
If you want to get good, race masters level. If you want to have a chance of a top placing, race regular categories.
As a new racer, however, you'll be a cat 5. They don't (well, not out here anyway) do cat 5/masters races.
What else ya wanna know?