I don't wear a helmet because I've been in more accidents with than without and my first helmet's still fine and dandy. Never really crashed on it / feels uncomfortable.

I grabbed the biggest one my LBS would stock, though, so I don't think I can give you much help on in-between sizing. Perhaps wear a beanie or you get one that's slightly bigger? That's what I do with my snowboard helmet. If smaller, you can remove some of the comfort lining of your helmet.