We've had the same kind of summer up here in Boyne City. It's windy every day. I have discovered that the first law of Murphy's Law of Cycling states that if you have a headwind on an out and back ride, you will have a headwind on the return home. We've got hills where wind swirls around who knows where, lakes/sun convection dynamics.....it's always windy. And yes, somedays where I was looking back to see when a jogger was going to drop me.
Just ride in the drops every chance you get and someday it wont be as windy and you'll be sailing along 25mph like a TdF'er without a care.
Ride on.