Mattm, I do not know if you remember the carnage in the 4s in 2010. There were a lot of bad crashes and it was heavily discussed on NCNCA forum. One of the possible reasons were large fields with racers upgrading too quickly from Cat 5. Then the Cat 5 fields were limited to 50.
Larry put out and email pointing out that this year USA cycling has increased the Cat 5 field to 75 suggesting but yet another reason to do the early bird series and work on skills.
Field size matters and the 4s in NorCal (100 racers), depending on the race, the speed (faster is better) of the race and who shows up, can be just as or more sketchy than the 5s. YMMV.