personally, I think there's a distinction between bicycle racing, and just riding a bike around the neighborhood. the latter's only a sport in the broadest sense of the word.
The problem is that the uninformed can't fathom competitve racing as a sport because their frame of reference is riding around the neighborhood.