I grew up in Redwood City, and I've spent a lot of time in SF. Now I live in Reno, and the Bay Area winters look like paradise to me. Barring the occasional really heavy rain--which as you said rarely lasts long--I'd ride year-round there. You never have snow, the tiny amount of ice you see twice a year melts off instantly and rain, in reasonable quantities, is easy to deal with.
The cold isn't a factor here all the time (it's not hard to dress for 25 degrees), but lingering ice and packed snow on the roads is a killer. Just get on the bike adn go.