Assuming your friend is not as obtuse as he sounds and simply having a little fun trying to bait you correct?
if so, just challenge him to a very simple contest.
You walk three miles with him and he rides three miles with you.
Assuming you are an avid cyclist as you post here frequently I am guessing he will not be able to stay anywhere near you on a bike but you will match him step for step on a walk.
Once that happens you then ask him how you can be in better shape than him if you are cheating