How much are sleeveless wool V-neck sweaters at Kmart or WalMart? Wear it right over a wicking T-shirt, under a regular shirt, and you can keep your core fairly warm in some cold temps. Get a cheapie pair of runing tights or the poly long johns mentioned above, and your legs should stay warm.
If you see the old style "union suit" long johns on sale, maybe at Meijer's, and they are not cotton, those are excellnet for colder weather. They won't leave a less insulated spot near your kidneys in the back.