Jeans and a t-shirt work great for me. I switched to a bag on my rack two weeks ago, and am wishing I'd done it sooner. I never had much in my bag, but the weight off the shoulders is wonderful.
I used to tuck my right pant leg into my sock, but then I got a bike with a fully enclosed chain guard.
I'm going 4 miles each way. If it rains I change at the office.