You are speaking the the 'Car-Free by circumstance' crowd. Or, impoverished cyclists. Making a blanket statement about this population is difficult, but I don't think the situation is as grave as you think.

A lot of the running talent from the Third World comes from the fact that running fast and far is often a necessity in these people's cultures. For hunting, primarily. For instance, I met an African guy who joined the Army to gain citizenship. He has literally killed lions with spears, and he has the fastest 2 mile run time I've heard of.

To ride a bicycle as a necessity for travel in one's daily business, I would imagine that speed is not as primary of a concern as it is when hunting large game with spears. So I don't think you're likely to find much world-class-level cycling talent among 'impoverished cyclists'. That said, I would wager that a good number of 'impoverished cyclists' regularly push themselves to go fast for fun, seeing how readily available this opportunity is. Also, they may happen to be in a hurry quite frequently, as is often the case while commuting. No matter what, you can say these people will have a good base to work off of, but I wouldn't go so far as to say thousands of 'Lance Armstrongs' just ready to win big.

Honestly, he was just plain faster than you. Welcome to Cat 6 - this should be a motivational experience. I'm sure you will see a drastic improvement in your performance with more riding.