Why not just have the women ride in men's fields when the field sizes don't justify having separate women's fields? I imagine most Cat 4 women could hang with Cat 5 men. Same with Cat 2/3 women and Cat 4 men.

You can still score them separately, but putting women in the men's fields might solve the field size and lack of competitive racing issues.

I agree with the complaint above though about how women's racing is being effectively subsidized by the men. I have nothing against women's racing and think in an ideal world it would be as competitive as men's races, but it's not something I am interested in helping to pay for, just like I would guess most women racers are not interested in subsidizing men's races.