I ride 3-6 doubles a year in similar conditions. First, many doubles offer drop-off bags where your jacket, light and other warm gear can be forwarded from an early rest stop to a late one or finish. Most of the times I carry my own gear. I wear a short sleeve jersey with arm warmers and knee warmers, and a reflective wind-proof vest. These are small enough to fit in my large saddlebag or jersey pockets. Short finger gloves work for me at 40F, but if it gets colder I carry wool liner gloves and use them under the cycling gloves.