Here's a comparison.
I don't drink alcohol. However, there was a group that I used to camp with who did drink alcohol. There was a food/spirits fund that we all pitched into. It was divided evenly amongst all participants. Was it wrong that my $ went toward the alcohol? Maybe. However, the food was almost exclusively BBQ & smoked sausage. There were members of the group who were vegetarians. They brought their own food but still pitched into the fund. The thinking amongst these folks, I think was if don't drink or eat meat, you should probably join a different group.
It's a whole package deal. Another comparison. My 25 high school reunion was a few years ago. They were asking for $150 a person to attend. This included a Friday night social, a Saturday picnic and a Saturday night formal. For my wife and I to attend, it would be $300. If we brought our three kids, $750. We were not interested in the social or the formal dance. We only wanted to attend the picnic. Unfortunately, there was no option for "pay for what you use." It was buy the package or don't. We didn't buy the package.
You are expecting to participate in the group ride. Does that not have a value?