Split the difference.

Pay attention if there's a significant delta between actual temperature and "Real Feel" (i.e., 10° or more), otherwise figure that whatever extra coldification is due to the wind (either self-generated or otherwise) will be offset by the heatification you create via physical exertion.