I wouldn't worry about it. My previous jobs I was riding a motorcycle to work and everyone was OK with me not participating in the 'drive to lunch' rotation. if anyone ever asked for a ride, i'd say "that's my wife's seat, you'll have to ask her"