Absolutely always wear one. Only time I may not is a quick test ride around the block after doing some maintenance.
Have had to call 911 twice for someone who did not wear one and they had crashed, seeing someone foaming at the mouth when they are unconscious is not a good thing to see.
I had a spill and broke mine once so I know I had hit my head.