I have to confess that last winter I did quite a bit of riding wearing an old pair of Dockers (I think they were 100% polyester though) with the legs secured with wool socks. Under them, I wore a pair of black Target polyester long johns.
On top, I wore a very light high-viz windbreaker that I bought at Sports Authority for $5 with another Target baselayer and wool sweater underneath.
Add cheapo gloves and a balaclava and wool scarf... good to 10F.
I'm dressed a little more stylishly this winter, but overall last year's fashion statement worked for warmth and comfort, even if it did look a little odd.
So before you head off to the Dollar store, you might want to check your existing wardrobe. You might find a gem or two.