What do you normally wear to work, for a walk around the neighbourhood, at home ....??? Wear whatever makes you feel comfortable.

On a hot day, I'll wear a sleeveless jersey.
A slightly cooler day and I might want a jersey with short sleeves.
Somewhat cooler and I might dig out one of my shortsleeved wool jerseys ... or my sleeveless jersey with a shortsleeved jersey overtop.
A little cooler and I'll throw a sleeveless jersey on under my shortsleeved wool jersey.
As the ride gets cooler I'll move into longsleeved jerseys, or maybe throw a vest on overtop, or armwarmers, or a combination of it all.
By the time it gets really cool I'll have a longsleeved polypro, a longsleeved wool, a longsleeved fleece, and a jacket.
If a ride is going to start cool and get warmer I'll wear a shortsleeved jersey with a longsleeved jersey overtop and plan to take the longsleeved jersey off later on.
If it looks like it might rain, I'll bring or wear my rain jacket.

Try out some combinations and see what works for you.