If you want something that looks like a fleece, any of the "Windstopper" stuff would likely do the trick, in addition to the other stuff mentioned. If you can find stuff that's only Windstopper in the front, or has pit zips and stuff even better.

I personally wear the Shower's Pass Elite 2.0 jacket for pretty much all my biking below 50 degrees (it's ok up to 60 degrees, but it's so windproof it's to warm after that). It's very, very windproof. In red, in the right size, I thought it was more than just "not goofy looking" - I thought it looked good. Sharp. Nice. It does look like a really nice looking rain jacket, though, so it might not work if you really, really want that "fleecy" look. In a size to large that I have to wear to get the torso length right it just looks "plain". Probably not what you're looking for, just thought I'd mention it - it looks fine off the the bike with jeans, I wouldn't wear something goofy looking, but I like the freedom of not having to worry about rain at all at those lower temps.