SteveE, the afternoon sea breeze is like a trade wind -- very predictable. It blows from a northerly direction, right over the bay. It picks up in mid/late-spring and tails off in autumn, and starts at 11:33am every day and dies down approx 7:11pm every night.
Luckily for me I live south of where I work so I benefit from the tailwind it provides. (At night, the winds reverse, but are much weaker than during the day.) These winds can make for a very tough weekend ride, though, and you'll find bikers tending to head north at first, then heading south for the afternoon portion of the ride.
During the winter months the prevailing winds will be from the south, but won't be as strong and not quite as predictable as those you're battling now.
You'll see visible evidence of this annual wind pattern by the route jets take to/from San Jose airport. For half the year (roughly summertime) the planes land and take off to the north, into the wind. They flip around during the cooler months. For months, the pattern will not change.
Think of the winds you're enduring now as invisible hills. You actually come to enjoy the challenge after a while. When they die off later this fall you'll be able to set world record speeds !!