I wear my cycling shorts for any ride over 20 miles and a soccer/running shirt to evaporate the sweat.
As for fitness rides: I wear cycling shorts and a jersey.
In general, I wear regular casual clothers for commuting and running errands (this accounts for the majority of time on the bike but ultimately less miles). I really find it enjoyable when I have to slow down as a means of not breaking a sweat. I think jeans, t-shirts, shorts, and sneakers are the perfect things for riding....just make sure you have a little sit-bone protection if you're going to be on the bike all day long.