Riding a bike at 15 mph in the winter is different than sitting on a motorcycle and riding at 50 mph... when I rode a motorcycle it got pretty chilly at 5C but cycling at -25C is less difficult.

Have you ever heard of a toque ?

They are nice little knitted hats that come in many weights and can be worn under a helmet... more technical versions come in synthetics and might provide better coverage for the ears and if you keep your head warm you have won a good part of the battle to stay toasty.