I've had very little trouble getting out when I need to. It's getting back in that proves problematic.
And I've got no beef with helmets, one saved me from hurting myself worse in an over-the-handlebars spill on my commuter. I still had a mild concussion, so I can imagine it'd only have been worse.
That said, I don't ride with one all the time because I've gotten used to having this headlamp rig and I've yet to figure out how to get it mounted on the helmet like I'd like. So I just don't wear a helmet when I've gotta ride home at night after work. I've gotta straighten that out, but A. I'm not sure how, and B. Time is killing me lately.