Base layers winter or summer are a good thought. Summer riding in Texas can be more than hot. When the temps in three digits the pavement can bake you from below. You need sun protection on your arms and legs and insulation.

I have to sacrifice speed by wearing light colored loose clothing top and bottom and the speed reduction is significant. If its really hot I might wear a light cotton T-shirt underneath and a pair of thin liner shorts under my spandex to protect against heat rash. My wife was looking at me funny last year when I had to get a ride in and it was about 102°F. I had my light weight loose cotton stirrup pants on and a long sleeve white shirt with a cravat under my helmet to protect my neck. She laughed and said you look like your going to go pick water melons in the valley. Yep... I got my ride in...

As to the OP: Yes I wear a base layer when Hot or Cold. I don't like to but I do. Kinda like my helmet...