When the weather gets cooler, I add the wool baselayer and arm warmers; sometimes the leg warmers. You can remove them when it warms up. Toe covers for the shoes. Headband to cover the ears.
Then it's the long sleeve jerseys with long sleeve baselayer and my Campagnolo bib tights. Cycling toboggan to keep the head warm.
Add a windbreaker to the above ensemble and full fingered gloves if it gets colder/windier.
Next up is a winter jersey that blocks wind. Colder than that and I use my winter cycling jacket and balaclava. Booties to cover the shoes and heavy wool socks. Good to go into the 20's. Doesn't often get colder than that here in the daytime.
Yeah, my cycling clothes cost more than your bikes combined. But that's what it takes to get ahead in the Poseur game.