Yes, anytime I ride I wear cycling gear. Each piece of apparel has its advantage over non-cycling clothing. I do not, however, have cycling shoes since I use Power Grips (straps) rather than clipless pedals.

But I do ride in a community where all kinds of riders wear full kit, but usually only the fast ones wear matching jersey and shorts team sets.