Yes.
Just got back from an early morning ride, temps about 38-40 deg. Most of me was fine, but my fingers were numb, even with two pairs of gloves on. Had to stop twice to warm my hands. Raynaud's is a pain, literally!
When I was a kid was always the first who had to stop making snowballs when my hands got too cold, although in those days I didn't wear gloves.