I get hardly any heckling, either way, but I FEEL less heckle-worthy when I wear loose shorts. Commuting I wear them for the pockets and comfort on my train ride.
I think nonbikers see skintight bikewear as pure pretentious foppery when worn outside of a competitive situation.
We know it's somewhat practical, but the bright colors and the revealing nature of the shorts gives the impression that bikers are saying, "Hey, look at me, I'm Ivan-freaking-Basso!" - much like if they were to put on a NASCAR firesuit to drive to work.
I suspect their contempt is similar to what I experience when one of those little Civics with the fart-can exhaust extensions and triple spoilers poots past me.