Hi,
same here in Maine. Potholes, sand traps, my commute in to work this morning was 35F with a headwind. I hadn't counted on a freaking headwind.
Anyway, I'd say the answer depends more on you and your bike than anything else. My bike is called a touring bike these days; but it greatly resembles the Schwinn Varsity I rode and loved over 30 years ago. The large-ish tires and mellow geometry survive potholes and suck up bumps. Which is the way a bike ought to be, IMHO. So my answer is, dig out the stretch thermal underwear; it's a good day to ride. Brrrr!