In chilly conditions, this is what works for me:

-- sleeveless jersey next to me
-- long-sleeved jersey
-- long-sleeved polypro
-- long-sleeved merino wool
-- jacket

I don't like wool next to my skin. It holds the moisture there and even the softest merino makes me itchy.

I do, however, like wool as my top layer because it sucks the moisture out there and dries off.