I don't have too much of a problem with this. Me and my 2 major riding buddies usually ride together and with no one else. When we do invite someone else to go with us it is usually someone that we all know. Those are also usually pretty short tame rides (the other person is usually new) so we just try to make the best of it and have a good time when there's a new face in the group. My situation is a bit different than yours though, you only get out with these guys once a week so this time's probably a little more special to you than it is to me.