I race a bit in CO (usually Cat 4 or Cat 4 35+ or Cat 4 45+) and also in another state which has the conventional USCF Cat 5 as entry level. It could just be that the particular races I enter in CO (Mt Evans, Iron Horse Classic, etc) attract some fast 4s, but it is fast - faster than Cat 5 in my home state. I did get a second place in a Cat 4 45+ last year, but man, I had to work for it. At the same time I was getting top 3 finishes at home without working terribly hard, at least until the last couple hundred meters.
With that said, just enter the race. What's the worst that can happen?