Check out the Salvation Army, especially if you are in a colder climate. I've picked up a few pair of wool gloves, hats, ear warmers, and a few base layers for less than $10. I wear regular clothing and as mentioned before, layer.

Usually an outer shell for weather conditions, an insulating layer for warmth, and a base layer for temperature control.