I’d say go large and put on winter gloves + liners. And, when possible of course, try to warm your hands / gloves before you ride. Haven’t tried hand warmers yet but I’m getting more interested about that solution. Because sometimes your hands can feel colder than usual (e.g. when your not too well) and gloves cannot always cope with that no matter which ones.