As I graduated from Clyde status, I found myself wanting the tighter-fitting upper garments. Partially because they fit better (just overall fit --with items in the pockets the tighter jerseys don't sag), and also partially because it feels good that I can wear it.
So for me it's about fit and some aesthetic. If I'm wearing a vest/jacket that flaps in the wind, chances are it's some inclement weather and I'm not counting watts but counting miles till I get home...