It depends on the type of cycling clothing. Spandex/lycra stuff is designed for road riding, where the only concern is sun protection. Baggy mountain bike jerseys and shorts are designed for fall protection, since falls are much more common off-roading.

The standard lycra kit isn't that much exposure in general you have two falling scenarios. Low speed crash where your hands get the majority of the scrapes, or high speed collisions/crashes where you're going to get pretty badly chewed up regardless of clothing.

And no, clothing won't make you faster, they may make it more comfortable. I can't imagine pedaling at 80 rpm with pants legs flapping in the breeze.