If riding 4-6kph slower is such an annoyance that it puts you off riding with a friend on a mtb, then don't do it in the future. For many people cycling is about a work-out and a challenge, for others it's a less "performance" oriented activity. No rights no wrongs, just different objectives. Personally, I'd love to have the fitness to ride at your relaxed pace of 35 kph for 100K.