I have had similar things happen. In fact, this week I felt very comfortable riding to work in 15 F but felt colder the next day when the temp was in the low 30s. I was dressed a little lighter the second day, but not inappropriately for the temperature. I chalked it up to the humidity and wind being higher, and my glasses did fog up easier the second day. It wasn't due to sickness.

When I am starting to get sick, I often find that my energy level is down and my pace much slower. Usually I don't feel any other symptoms yet and wonder why I had such a slow ride, then the next day I get a sore throat and a cold starts settling in.