I'm just starting my second year of racing and my biggest suprise in year 1 was how critical the 'non-fitness' stuff is.

Yes, you need a decent level of fitness to be in the race (especially a hilly one), but I can attribute almost all of my failures last year to lack of racing experience. Most of that was crit racing, but I found the same issues in a couple road races (but to a lessor extent).

Early in the season last year, I just focused on racing as much as possible (at least once a week) and forgot about any super-structured training. I'm not saying that's right for you, but I feel I'm in a better position now to leverage my fitness (or lack there of) because I've developed some basic skills. The fitness alone isn't much good unless you're one of those freaks that can ride off the front and drop everyone.