Is the question a matter of infidelity? And if so, is it a matter of adultery, or lust/coveting (mental adultery, to coin a terrible phrase)?

In my own experience it comes down to a matter of trust. And not just trust in your spouse if you are the one sitting at home and not on the bike, but trust in your spouse and the friend/riding partner.