Winds do keep me inside. On windy days I choose a route which will take me into the wind on my way out, and let the wind push me back when I return. On rare occasions the wind has changed mid-ride, which sucks.
I was once in a race, and had broken away with a Venezuelan rider. The wind was god-awful, and we (mostly he) worked hard to push through it. We thought at a any moment we would be overtaken, and worked hard to keep in front. It was the most agonizing ride of my life. But the main group never caught up, we placed 1st and 2nd. The next morning I was so sore I couldn’t lift my leg over my bike. But it was my best finish that season, and worth the pain.