Once you realize that all of the "roadie" gear is designed to help you ride better, longer, and faster you will begin to use them. They are tools. Pick the right tools for the job.
Riding 50 miles in blue jeans with a thick seam located right on your perineum is not a good idea. Fighting wind in water-absorbing, floppy cotton Tshirts isn't a good idea either.
FG riders have more in common with road cyclists than not.