Anecdotal, but I haven't had anything more than a minor cold for the last 3 years, and even those have only happened once or twice per year at most. I do think commuting year round in all weather builds a strong immune system.

That said, if you do get a respiratory infection, I think it's better to rest than to ride and draw these microbes deeper into your lungs. No need to turn what would be a 2-3 day cold into a month-long bronchitis.