Maybe try something thin like this: Amazon.com: Cycling Sports Face Mask Cool Fashionable Ultra Thin Balaclava (Black): Sports & Outdoors

I use one of those when it's below 10 degrees. 40 degrees I just rock a headband and sunglasses. No issues breathing through it either way.