I tend to wear a helmet for serious riding, except if I am climbing and chances of a high-speed crash are low. I have had crashes where helmet saved me some skin or even brain tissue.
At the same time, I do not feel I have to wear a helmet if I am going to the bakery. I think making helmets mandatory was one of the dumbest things that was done for cycling.