I try to look reasonably presentable when I'm riding on a tour, and especially if I'm going to be going into businesses where there are other people.
Rowan and I found these great wicking tops at Decathlon in France. They don't look like jerseys, but they do what jerseys do. And I often ride in zip-off pants. You might find similar attire in places like REI.
So here I am riding through France:
http://www.machka.net/pbp2007/2007_Post-PBP.htm
http://www.flickr.com/photos/1430288...301323/detail/
If I were you, I would not bring jeans and cotton T-shirts on a long ride. They are heavy and take forever to dry. Look into some lighter clothing options like the zip-off pants and wicking T-shirts. They look all right ... I think they create a look of someone travelling ... and they are easy to take care of.