I don't find 40-50mph gusts too troubling, though they do make for a lot of spray when I'm riding I-90 across Lake Washington. Steady gale-force winds are depressing if they're headwinds, invigorating from behind, and annoying from the side, especially mixed with rain.
When I lived on an island, winter winds over a moderate gale would sometimes cause trouble on low-lying roads, nothing like riding through cold saltwater up to your ankles to make someone appreciate that hot shower at the office.