Sounds like it's time for some experimentation. Try warmer gloves and a little less insulation on the core. This might send the signal to your brain to increase the metabolic process slightly.

Also, sometimes when I go out and am warmly dressed and wearing gloves, even warm ones, my fingers will feel a little bit cold. Some of this is just because your fingers don't have much ability to keep themselves warm. There are no muscles in your fingers and not much metabolic activity. So they don't generate as much heat as the rest of your body. So they need more insulation on them than you might think to keep warm.