I got myself this helmet cover:
MEC Drencher Helmet Cover (Unisex) - Mountain Equipment Co-op. Free Shipping Available
Highly recommended, fits my helmet like a glove, and keeps way more heat in than I thought. Just the helmet cover with a headband or something to keep the ears warm is good down to -5 C for me.
Anything colder and I add in this balaclava which is thin enough to fit under my helmet:
Outdoor Research Ninjaclava (Unisex) - Mountain Equipment Co-op. Free Shipping Available
Having the helmet cover makes a huge difference - stops the chill from getting in and actually helps the helmet keep in a surprising amount of heat, which is really nice.