Most days I never see another cyclist on my route, so if I'm wearing what I'm wearing to impress other cyclists, I'm doing a really bad job of it. I thought I was doing it for practical reasons - bright colors improve visibility, wicking material is more comfortable when you're sweating, tight-fitting clothes improve aerodynamics.

Tsl makes some good points, and I'll add a low-budget perspective. You could start out with some of the cheaper shorts and jerseys at Nashbar - they might not fit as well or last as long as the pricier stuff, but they'll give you an idea what it feels like. In colder weather, you can supplement with tights and a thermal undershirt. I got my tights at Goodwill and my undershirts at Sierra Trading post for less than $5 per item. A windbreaker/shell makes a huge difference when it's cold, as does a balaclava, even a lightweight one.

If you'd prefer to stay away from cycling-specific clothes for whatever reason, I'd still make two suggestions. First, funkiness can be pretty much eliminated by riding in freshly-laundered clothes on a freshly-showered body, and then changing into different clothes when you get where you're going. Second, avoid cotton (for your riding clothes) at all costs. Doesn't wick, and doesn't insulate after it gets damp. If you want natural fabrics go with wool or silk.