I've toured on a few of Adventure Cycling's more popular route segments. I find just the opposite is true. I enjoy meeting the occasional other bike tourer along the way. The small towns that have experience with bike tourists seem to relish the business and are better equipped to offer things a bike tourist needs.
Cars get used to seeing bikes on popular routes. That improves safety.
The more bike tourists, the merrier. There's plenty of room in the pool for everyone.