Getting back to the OP's hypothesis, actually, going out in the cold would cause the bearings to loosen up. This is because the axles are usually made of steel while the hubs are usually made of aluminum. Since aluminum shrinks more than steel as they cool the bearings would loosen.

Shrinkage???
https://www.youtube.com/watch?v=pm3F9piwnTU


Just for the record, I find I am fastest when the temps are 70-80F.

Think of all the cold air you're sucking down. Your body has to devote energy to staying warm.

Try a breath warming face mask. I use a Psolar.

On the plus side for some, I suppose, is the fact that a lesser degree of intensity can produce more calorie burn.

I ride a 50 pound extracycle in the winter...

It runs smoothly at any temps (synthetic grease ftw) it's weight does affect my top speed a little but I am still faster than most winter cyclists because of the incredible stability of the bike.

The biggest impediment to speed in the winter tends to be wind resistance as the air is denser and we are less aerodynamic... that and riding your bike at 40 when it's -40 requires snowmobile gear to stay warm.

Many athletes perform at high levels in the winter... biathlon and cross country skiing come to mind.

The air is about 15% denser in the winter than on a hot summer's day (O vs 40 degrees C).

The drag is probably partially offset by the fact that you can breathe more oxygen with less exertion of the chest muscles and diaphragm. As 65er has pointed out, the intercooling can boost performance although I don't know if XC skiing is as fast as cycling, which would make the air resistance loss less important than the cardio boost in those sports.