My favorite so far is a Giordana FRC jacket. With suitable base (including an awesome old uniform wool sweater), I've been quite comfy below 20F.

If it's just chilly, there's no need for anything but a regular jersey beneath a jacket - just get something to stop the wind. But if it's cold, you definitely want to wear something more under ANY jacket.

Also, while you don't want a long, flowing muffler/scarf, some extra bulk around the neck can do wonders.