I don't think this is even remotely true. Try going down to a city park and getting into a pick up basketball game. Now try it out of shape and wearing penny loafers and see how many times they pass you the ball. Isn't this similar to what the OP did?

Ever just walk up to a bunch of soccer players getting ready to start a game and see if you could join them? Maybe hop into a volleyball game or a softball game? Would you usually be welcome? Cycling is one of the only sports where a complete outsider is welcome on most rides. Yes, there are a few "elitists", but there are far more friendly and welcoming bike riders and racers.