You're all grown ups, so you can decide for yourselves. For me it's not the length of the ride, or type of riding that affects my wearing a helmet. A helmet is for an accident you won't ever be able to predict.

Whatever you decide, don't dangle the helmet from your handlebars. I see that all the time & makes me laugh. What is the helmet doing there? Is it a bike ornament? Protecting the handlebars? Or did the rider feel some pang of guilt & decide to take the helmet along, but not to wear it. Bizarre...