Agree with "tailwinds".
There was an evening a couple months ago that was just perfect. Great tailwind!! Everyone was out on the Cycleway going for KOMs ... and I accidentally picked up a QOM myself!!
IMO, if clothes make a difference, it will be very minor. Give it a try and see.