I am another who does not care for the "start too cool" approach. 1) my knees and other old injuries don't like it and it is setting me up to aggravating those issues the whole ride and later and 2) if I have to stop (for a flat say) it is quite likely I will not be able to get my body temp up, both slowing my ride and (for me) dramatically increasing my chances of succumbing to a cold or flu or whatever has been around.

I wear enough to start out warm, but clothes that can be easily removed and stuffed into pockets or a stretchy sock I secure under my tool bag with a toestrap as I warm up. This means that if I have to stop later or the weather changes, I have a good chance of getting home and not getting sick. (This comes from my racing days. Training was all important. Loss of training time and miles - to be avoided at all cost. If that means having too much clothing to be stylish, so be it.)

Ben