For your body:
- An outer shell wind/water proof jacket. I agree with the comments about the zip-pits variety. I use the same shell nearly all year long. It is just a regular wind breaker so I also use it when traveling, and going out.
- water proof pants.
- wicking base layer top and bottom. I highly recommend the merino wool variety rather than the poly-variety. Merino wool will be more expensive, but you can wear the same set all week, with the poly-variety (Under Amour etc.) you will need several sets. Now is a good time to buy as the stuff will be on sale.
- Wool socks
- Water proof shoes.
- A balaclava and hat that fit under your helmet.
- Wind proof gloves.
- Warmth layer - fleece/sweat shirt/etc.

I tend to not buy bike specific clothing. I can use all of my biking clothes for other activities, which takes the sting out of the cost and they usually are cheaper than the biking variety. Another trick is to go to Costco's or other big box stores and get merino sweaters cheap. Though it can take some looking. I do a lot of my shopping for these things at outdoor stores, army surplus stores or utility work stores. Let me reiterate, if you are going to splurge on anything, make it the merino wool under layer.

For the bike:
- Lights
- Fenders
- Tires that can run at a low PSI without pinch flats.

I also carry cab fare and a cell phone at all times when riding in the winter, because I am much less tolerant of the exposure time of changing a flat.