The worst part for me is dealing with the cold once I get to my office. I'm usually underdressed when I go out for lunch since you can't wear too much when you're biking (I know, bring/leave clothing at the office, but that only takes you so far into the winter unless you have two winter coats).