I'll wear street clothes for short trips or when going someplace where bike clothes aren't appropriate and changing isn't practical.

Sometimes normal activities call for clothes other than what you'd wear around the house or at work. If I'm walking to the store and it's raining, I put on a raincoat. If I'm playing in the yard with the kids I might be wearing something completely different than if I'm having lunch with my boss. All of these are normal activities and often people dress differently depending on what they're doing.

I don't feel the need to make an exception for cycling. It is a normal activity for me but it's not the same as going back and forth to work in a climate controlled bubble that requires virtually no physical exertion.