What do you wear on top in 30-50ºF weather?

I'm looking at the Gore Phantom 2.0 jacket, but it's spendy. Is it worth it?

Or better to layer with thermals? I rode last weekend (road cycling), it was windy and 40º. I was struggling between a regular polar fleece jacket which was too hot and a long sleeve base layer which was too cold (meant for skiing).