I went on a group ride with the bike club once and rode alongside a woman that turned out to be a psychologist. We discussed negative motorist reactions to cyclists (including rage) and her opinion was that a lot was due to envy. Not just of the fun, but of the health benefit.
As for "How far did you go?" I usually get the dropped-jaw reaction when I say ten or twelve miles. I don't think anybody yet has expected me to go more than a mile.