Arm warmers, leg warmers and a lightweight shell jacket should do it. You might consider a base layer (wicking t-shirt) as well. Everything in the list except the wicking t-shirt are easily removable as the temperature rises. If the temperature is closer to 32, you'll want full-fingered gloves and maybe a headband for your ears.
However, the problem around here (central Texas) is not the temperature, per se. It's the fact that you don't have any chance to grow accustomed to the lower temperatures. We can have 80 degrees in the morning and a 50-degree drop in the afternoon when the cold front blows in.