Yes but you have to race smart and choose your races carefully.
I'm 5'7", 170-175 or so, not skinny. I'm pretty sure my % fat is in the high teens. I'm not a strong Cat 3 but I can place or win in select races. Most group rides turn into extreme sufferfests for me though.
Ideally I'd be lighter but that involves efforts I just don't want to make. I have a few weeks to my next race so I'm trying to eat less (I normally eat relatively healthy food so I just need to eat less of it), ride more (long hours on the bike melt off pounds for me), and see where I end up for Memorial Day weekend. Without really dieting I got down to the low 160s one year, for April, so it's possible. And I was working and in a relationship at that time too.
I was best when I was single, extremely poor, and literally couldn't afford 3 meals a day (during the week, okay, but on weekends I'd be looking for any stashed cans of food - so sometimes dinner was a can of corn or tuna, but not both). I was in the low 130s, about 4% fat (7mm in 4 pinches), and very strong.
You'll be fine. Only thing you won't have is the ability to say "Yeah, I need to lose another kilo" and raise up your shirt to reveal a perfect 6 pack and say, "See what I mean?". This sort of pre-race psyc out will not be part of your arsenal.
cdr