I rode 40 miles a couple of weeks ago in jeans and a cotton tee. Not a big deal.

My biggest problem is with the cotton tees. I can ride in jeans or jean shorts or whatever, but I much prefer a wicking shirt. I can't justify springing for a shirt that costs more than $60, so I pick up $9.99 "duodry" shirts at Target. Very comfortable and I will wear them even off the bike.

That said, I do think cycling shorts are more comfortable than jeans, and if I intend to go on a long ride (the 40 miler a couple of weeks ago was spur of the moment once I was on the bike) I'll wear cycling shorts underneath something else. I like to have the option of going in places, because I'm never out on just a training ride, and I don't like walking in places in skin tight shorts.