It happens to probably most of us. Even if I'm wearing a normal pair of shorts and t-shirt while on a mountain bike, I'll still get "Lance" hollers.
Honestly, I think it's the helmet that does it more than anything. Whenever I carry my helmet with me (or leave it on) into a store, I always get some sort of a "how's the ride goin, Lance" from the cashier. If I'm in full gear and leave the helmet with my bike... nada.