Just tell 'em you're thankful that your gut's not large enough to qualify as a Harley rider (it takes a Hog to ride a Hog).
To add injury to insult, find a nice twisty downhill and challenge them to a race - those big ole Harley's can't corner for sh*t, and it's not too hard to outrun them on downhills (I've passed Hell's Angels going down hill in Colorado).