I don't know how you feel about thrift stores, but I've found the best trousers for cycling to be wool dress slacks... I can pick 'em up in many colours and whatnot for under 10$ usually. New ones are likely to be had in all sorts of costs and places too. I rode most of the winter with them and just cyling shorts instead of underpants... splash pants that I already have in heavy rains or winds, but anything warmer than -15C was just the slacks. I've found lighter ones to wear in warmer weather too. They didn't acquire any funky odours with reasonable use though they do need to be ironed but that's not really all that difficult. They also look really sharp I tend to wear linen in the summer but that isn't as easy to find on the cheap. I've done 10-50km rides with this slacks and bike shorts combo. When I worked in an office many years ago, I would just duck into the washroom and change out the shorts and stash them away in my bag.