I'm sadistic and enjoy riding in somewhat challenging conditions. The hardest part of riding in 25-30+ mph cross winds is when you have to lean/steer heavily into the crosswind to stay upright, and then the wind suddenly stops coming from that direction, or even changes/swirls to the opposite side, and now you're leaning heavily but there's no longer wind resistance, so you quickly swerve in that direction. I had to abort a ride up Mt Evans this year due to fast 25+ mph crosswinds at 13,000' that were blowing me into the car's lane of travel due to this. and I just couldn't counteract the crosswinds effectively enough to hold my line on the shoulder. I like being spanked, but not spanked to death.

Headwinds on the other hand are just extra resistance, like climbing a hill you can't see. And then there's HEADWINDS that can turn a bike into a kite...