Maybe it's something about where you live or the people you interact with. I rarely get any comments like that. Even the ones that don't know what it is just say something like "Oh, that seems hard!" or whatever when I explain it. I don't know if I've ever been challenged like that when talking about riding fixed.