OP's question was should he ask "Mind if I follow at a distance?" Am I alone in thinking that's just going to create confusion as to whether he is or isn't joining the group? If all he's doing is following at a distance, chances are pretty good the group isn't even going to know he's there or, if they do, they would probably just assume he just happens to be going the same direction. Either way, it won't affect their ride in any way. If he wants to ride close with them he should ask, but otherwise, why would he need or want their permission?