I think you have other problems... ditch the uber-cyclist in tights crap - that for sure won't fly in your type of environment. Wear shorts or pants overtop. The hi-viz stuff doesn't need to make you look like a crossing guard. If that is all these guys are teasing you about then that should fix it.

Obviously you also need to stop getting so worked up about it or you won't last long.