I hate the wind, but I don't let it stop me. (I'll feel less bad about it when my power meter gets in...) I've been out a few times this year with gusts of 40 or 45mph, and in certain areas gusts get to be continuous. Makes a "short" ride awfully long when you have to work to descend at 10-12mph that you can normally coast at 30.
All that work into the wind makes you stronger, just like hills.