if you hold steady they won't pass you because you haven't slowed down. unless it's a race style ride, it's generally bad form to speed up the pace of the group, so if you hold steady you will stay in the front. if you're cooked, then you can't hold your pace any longer, hence why you're pulling off. so tell that guy that you're not am magician and when you can't keep your pace anymore you will pull off which is the proper way so the next guy can start leading without having to surge to pass you.