If you're comfortable riding in 5 degree weather in a speedo, that means you generate a lot of heat and tolerate the cold well, it doesn't mean a speedo is warm.

Since everybody is so different it's not like you can just read what works for somebody on the internet and hope it'll work for you. Lots of experimenting is in order.

Usually the key is layers because that's what's most versatile.