I had two sets of tandem friends and it was a blast drafting behind them. One set was smoother and other a bit more erratic but strong. But there was the time during the last ten miles of a century that one of our group actually came back on the course looking for some of us. He was incredible strong and consistent. I drafted that last ten miles and it was dream come true for a wasted rider - me!
So, if you can be trusted and you trust them - have at it with permission.