I've been doing it a few years so most people at work don't comment unless it's really cold and/or snowy. However, on the elevator on the way up to my office it's a different story. It starts on the cooler days in October and continues all fall.
Them: "I bet it was a little cold on that bike today, eh?"
Me: "It'll be colder in January"
Once January rolls around people just stare, assume I'm nuts, and don't bother to question me on it.