I regularly commute in bike shorts with chamois and Dickies work pants over them. Today, it was 45 degF leaving the house and shorts + dickies was perfect.
I'd hesitate to wear heavier 100% cotton stuff like Carhartt workpants and the like. 65/35 poly/cotton blend and weight of Dickies (Red Kap, Wearguard, Sears brand, etc.) works for me.