If it's daytime and there's sun around, I just go with a long sleeve jersey and shorts, and yeah it's cold. If it's a night, to be honest, that's what the trainer is for!
I did see a merino smartwool long sleeve undershirt on sale, so I ordered it, but it hasn't arrived yet. Hopefully that'll be enough with a jersey and windbreaker at night.