The problem has nothing to do with "avid cyclists" The fit and healthy have always motivated society, the problem is being a fit and healthy cyclist that is too lazy to go ride in places out of the way of car traffic. All sports with avid participants have a designated place to do the sport with the exception of road cycling.

On the other end we have the perception of people too poor to drive so they bike, viewed sort of like the homeless begging for money on a busy street corner... "Get a job, be somewhere else." This perception may be different around some universities but otherwise we are out of place no matter how we dress.