I don't bike in work clothes. Not TO work anyway. Sometimes I'll ride home in them if I don't feel like changing.
In the winter it's too cold. Khaki's don't keep me from freezing.
In the summer it's too hot. Khakis will cause so much sweating I will smell like a gym locker room all day.
In the spring temperature isn't a problem but it's always wet. So I'll get my khakis dirty.
In the fall.....I dunno I guess I could. But I spend 9 months of the year changing clothes when I get to work so I stick with it.