That's why they call it "active transportation," and it's a double edged sword. If you were in a car, with the windows rolled up, and the radio blasting, people wouldn't see much of you, and you wouldn't hear anything if they did yell at you. On a bike, there's more interaction with the world, because you're more a part of it.

I've seen a lot of lady roadies in lycra lately, now that it's getting nicer out. It's not so difficult to see why people would be attracted to them. They tend to have well developed leg muscles... Also, there's a rule: being on a bike makes a lady ten times more attractive than she'd otherwise be.