Just get a wind jacket? I have just a plain, very thin jacket that keeps most of my body warm with good air protection.
The only problem I ever have, when descending below freezing temperature, is my feet getting cold. Perhaps mountain bike shoes don't block the air well.