I do have a down jacket and a down vest that will get hauled along on tour, in the colder months the down vest lives in the handle bar bag for a quick core cover when riding in cooler weather. I gets layered under a wind proof jacket. Hard to beat down for the warmth to weight to size ratio, but it does need to be kept dry. I almost always lug a wool sweater along too.
Aaron