I am a bit surprised at their lack of maturity, and Management allows it to happen. I guess they have not been to any HR Training that is mandated due to lawsuits and payouts for stupidity. I would still wear the shorts, why should you have to change your shorts or take your jersey off to suit them, when they are the ones in the wrong? I would keep wearing what I had been. If it bothers you time to take it up with HR. HR has no choice but to act. You said the office was 90% male, and I would guess the other 10% are females. Not that it matters, but they can complain as 3rd party, and claim injury due to their behavior. I guess I work in a great environment, no one says anything about what I wear, nor do they say anything other than how was my ride.