I just yell back "No!" or ignore them.
When I was stopped at a railroad crossing recently, I had this strange conversation

Lady -"Why do you ride in the street?"
Me -"Where else would I ride?"
Lady -"On the sidewalk" (with an implied 'Duh')
Me - "Well, as you can see, there's no sidewalk here. Most sidewalks are not safe and sometimes even illegal for bikes. Plus, you cant really go very fast on the sidewalk"
Lady -"Oh, just wondering"

I would like to think that was an educational moment for her than might change her attitude about bikes on the road, but that's probably just delusional.