If you have it, don't mind riding in it (Unless I am doing a self-contained tour where there could be cold, wet weather, I usually skip the rain pants because I don't like riding in them.), and space allows, I would take the full set. As we saw last year with Irene and Lee, anything can happen at any time. If you have capacity, I would pack for all potential conditions save for maybe snow. Next week, I will be heading to Cycle Oregon, which is a week-long supported tour. It could be hot and dry every day. It could be very cold some mornings. One day we are goong to Crater Lake, so it could be snowing. (When the ride went there a several years ago at the same time of year, it was 40 and snowing up on the Rim Road.) You are allowed one bag up to 65 lbs. That's more then enough capacity to cover all the bases.
If arm warmers and leg warmers might be in order, so might a pair of full fingered gloves or polypropylene glove liners that you can wear under half gloves for those chilly mornings. I did a tour in northern VT in August a few years ago and used my arm warmers, leg warmers and glove liners on a few chilly mornings.