I know it's been a cool summer but I haven't even started thinking about fall yet let alone winter!

Everyone has different levels of comfort with regard to being hot or cold and then it also depends on how long you'll be riding and at what intensity. My rule of thumb is that if I'm warm enough when I leave the house then I've overdressed. I aim to be chilly when leaving knowing that my body will start generating lots of heat within 5-10 minutes of physical activity.