Start with base layers. The Champion stuff at Target is inexpensive but works as well as the high-dollar gear.

Get a vest, preferably neon green for visibility. Get arm and leg/knee warmers. This will make all of your jerseys and shorts usable in cool weather. Long-finger gloves. Buy gloves in person to make sure they fit. Nothing worse than gloves with fingers too long. For your head, get a fleece skull cap, and balacava for really cold weather. Toe covers for your shoes, and wool socks.

Next get tights without the pad. Just wear them over your shorts when the weather is cold. That way you don't have to wash them every time you ride, plus it gives you another layer over your legs on cold days.

The last item I would buy is a winter jacket. See how far you can get layering before you do that. However, some of the new winter jackets are amazing - warm but breathable within a certain temperature range. The last thing you want is to overdress because that will make you sweat and get you even colder.

Resist the urge to buy black jackets/jerseys because they look trendy and hip. They make you almost invisible on the road, particularly in the dark. If you ride in the dark at all, look for yellow or neon colored jackets/jerseys/vests for winter use.