I started out with sports tops and baggy shorts. Then I went to padded "under" shorts (i.e. the type you can't see). Now I'm on bike jerseys and lycra shorts/bibs like everyone else.
There's a reason people on bikes wear this stuff and it's not to look good. It's comfortable, it wicks sweat (one thing I learned using trainer in the basement, I sweat like crazy while cycling), it allows you to go long distance without chafing. The back pockets in cycling jerseys are so useful, that I can't ride now is just regular sports tops, I have too many things I like to stuff in those pockets!
A friend of mine challenged me on my choice of clothing once, saying that I was just wearing it too look "cool" (if you can believe that, as if my beer belly looks good in a cycling jersey), he's a jogger so I asked him why he didn't go jogging in Doc Martins and jean shorts.