No doubt, Fall and Spring are my favorite seasons (followed by Winter, but for other reasons). But one thing that drives me up the wall is all of the extra clothing I have to bring.
When I leave in the morning it's between 40 and 60 F (it's been an unusually cool August). By the time I leave for home in the evening, it's often 80. This means I need to bring cool weather gear in the morning (must keep those knees covered!) but keep shorts and a lighter shirt for the ride home.
It's not a big deal really, but on the days I forget to bring shorts for the afternoon it can be pretty uncomfortable. Those are always the days with the widest temperature swing.