I wear shorts under my Carharts. They don't get in too bad of shape. The main problem I've had is with jeans ripping out in the crotch because they ride lower on my hips than they would normally (that lycra is some slippery sh*t) and I tend to clear jams out of my machines at work near the floor or if I need to work on them. I sit kind of weird when I want to be that low - pretty much both ankles at my hips sitting with the bottom of my leg out. I dunno why, but a lot of other people can't sit down like that. But I've ripped the crotch out of two pairs of jeans so far since I started wearing the shorts and they started slipping. (I don't bother to take them of for work BTW - we have uncomfortable chairs).
As far as the boys go, by ass is in worse shape sooner. But I made sure I've got a good angle on my seat so my boys don't have difficulty. The damn seat I bought when I was drunk doesn't help that (Kiddies - never buy a carbon fiber 135gm saddle that you have never sat on when you are drunk. It is not a good idea, and may lead to bounced rent checks)...
The ride I went on sat had a fair amount of people in bike garb, a couple even with tights and jereseys... Or jerseys and cut-off pants that probably had bike shorts underneath.
It really doesn't matter at all what you wear. (except maybe a dress, that would be way to easy to catch in the chain unless it was a miniskirt) Just wear what is comfortable to you, and ride how you want to ride. I'd give props to someone I saw that I was able to identify as a roadie by outfit that was riding a bike that I knew was a sweet fixie....
That could just be me though...
BTW - Glad to see we didn't completely scare you off. Riding fixed is something that will help you out in every aspect, not just riding your bike. If you drive a car, I can pretty much guarantee that within a few months you will have a much better understanding of traffic, and how to predict it's movements...
It's all faster slower, never stop, not fast to the light, then stop. Not a re-incarnation of the car where you are the engine, but a way of thinking about motion that trancends everything you ever thought.
If the standard issue is what works for you, keep wearing it.
As long as you ride we (or at least I) don't care....