Layers---that's been said. I wear what I have: a lightweight Patagonia capilene short-sleeve shirt, over which I layer a Patagonia long-sleeve midweight capilene zip mock turtleneck shirt or a 150 weight fleece, plus a very light-weight yellow outer rain/wind jacket. I use the capilene for hiking and cross-country skiing, too.

Booties (outside shoe) or goretex socks (inside shoe), something for the hands, rain coat, rain pants optional, wool socks.

I've worn goretex socks and liked them alot; my feet stay warm and dry. Had to switch to neoprene booties (per bike fit, I now need to put a 0.25" lift in the shoe to lengthen my leg, leaving no room for goretex socks). Neoprene also works well for me, but, your feet can get wet with neoprene. Never had wet feet with goretex socks.

I rarely wear rainpants, but always have them with me October to June, just in case of a downpour. A pair of thick tights is more comfortable, kept me warm enough even though I got wet in the rain. Only in heavy downpours did I wear rainpants.

I wear my specially-designed-for-bicycling hand-knitted wool mittens in all cool weather; the thicker ones for winter, the thinner ones for spring and fall. The wool kept me warm even though it's wet. They dried out enough during the day to wear home. If heavy downpours are forecasted, I'll have an extra pair of socks and mittens with me. Gloves don't keep my fingers warm, others do fine with gloves.

I wear wool ankle-socks year around, switching to heavier socks, the next step up being lightweight hiking socks, in freezing (30s) weather.