If you tell him you're going to ride with him, then ride with him. That doesn't mean you can't show him a thing or two, but stay with him and keep your ego in your seat bag. If you're not willing to do this, then decline. If you go out with the hidden agenda of dropping him so you can make yourself seem superior, then it's not worth anyone's time.

Oh, and if he's on a road bike, take your road bike.