OK. So this isn't exactly a crazy (or new idea). Up until now I rarely rode in the winter, so it didn't come up that often.
My biggest problem, is that I'm just about at the upper limit for the medium Atmos. Once I layer up on my head (beyond a Head Sweats), it doesn't fit that well (I suppose I could shave my head, but . . . ). Like I said, it wasn't a problem when I primarily rode in the spring, summer and fall. But now that I'm commuting in the "winter" (yes, I know, California winter is a nice spring day in many other parts of the country, but humor me), I occasionally wear some form of head gear to keep my ears from falling off.
Thanks for the feedback. Now, to find a poorly ventilated helmet that fits a little loosely . . .