Sure.

For me personally, I'll wear regular clothes or workout/cycling clothes depending on distance and intensity. When my commute was less than 2 miles, I wore regular clothes. When it got to be 6 miles each way I decided I wanted to make it part of my exercise regimen so I wear cycling shorts with a wicking t-shirt. If I'm going on a group ride after work, I substitute a cycling jersey for the shirt .

I don't worry that my clothing choices might make commuting seem less accessible to somebody else. There are plenty of people who ride in normal clothes and I figure any adult is capable of making their own decisions when it comes to what to wear.