They're your hands, man. If your hands are cold, it's time. If they're not, it's not.
I have friends that put long fingers on at 52. I don't need them until somewhere in the 30s. Everyone is different.
Throw the short gloves in your back pocket. If your hands start to get all sweaty, switch.