You really have to try. Get layers, get stuff that is good with sweat. Sweat is liquid water, which will insulate less than the air in your insulating clothes.

Sometimes at 75°F I ride in shorts and T-shirt and I see cyclist on road bikes with full arm and leg length lycra whatever cycling clothes (looking like they are on the way to the winter TdF). Everyone is a bit different.