If I am going out just to ride I wear long johns under the shorts I ride in, smart wool socks, a synthetic zip up, with a cycling jacket over that, plus gloves, and a scarf. The gloves and scarf will go a long way to keep up warm. If I am using the bike for transportation than I wear smart wool socks that come up to the knee, jeans, a hoodie, and an east german army navy surplus zip up I found for $15, also the gloves and scarf; if it's really cold I'll also use long johns.

The trick is definettly a combination that breathes, is comfortable and allows for a full range of motion. And with the layers you have the freedom to unzip, or take off a layer that is keeping you too hot, you want to avoid being cold and sweaty out in the elements.