Don't be embarrassed, nobody is perfect. I find that the opposite often happens, I have thinner riders encouraging me. Or that I simply ride stronger than they do and they won't say a word. I wear biking shorts and I think cyclists would laugh more if I wasn't wearing them.