Sounds like possible circulation to me, so get that checked out first. At 41°F, I can still wear shorts with a long sleeve shirt and light gloves.
If they dont' find anything, keep your legs warm. When it gets too cold, I have to add to my legs even though they aren't cold. Gotta keep the pipes warm so to speak.