I have never in my life felt that my ankles were cold. Toes yes, but never ankles.

(My solution for cold toes is to HTFU because work is only about 5 miles away, but if I were to ride longer than that in the cold, I would use plastic grocery bags between socks and shoes. Actually, since the free bag ban passed, I don't think you can get those thin plastic bags anymore, all they have now is the thicker, reusable bags you that cost 10c each)

I wear shorts in all temperatures I get, which stretches down to about 30 a few clear winter mornings per year.