If you don't mind everything that is on the road; oil from cars, grease from dead animals, urine from who knows what, spraying up onto your legs and possibly in your face, whenever you ride when the road is wet, they're not all that important.

I personally ride on gravel road about 8 miles a day, so what splashes up from the road is VERY different than the rain; even with fenders and a mud flap, my shoes and the bottom few inches of my legs are totally covered with sand and mud, but above that it's just wet.

I don't really know what could even be "irritating" about them. I've always had them on, and they're just part of my bike. I don't think I'd know if they were on or off except when it was wet.

If they're noisy, they're not attached right or something; maybe they're just badly designed. I've had Freddy Fenders, then SKS and now have Planet Bike Hardcores, and they were all fine until they broke (once due to a stick blowing into the spokes during a windstorm, once due to vibration from the gravel roads eventually just shattering the mounts over 10000 miles).