We don't have that problem at our office. I'm on the thin side, but so are most other people. However, I don't smoke and half of our people do.
Besides, they know I'm not doing competitive cycling. They see me arrive at the office (or leave it) in street clothes, even with a suit and tie, and helmet, rather than in cycling clothes, so maybe they don't take me too seriously.
On the other hand, they see I'm serious (or nuts, it depends), when I cycle through snow storms or heavy showers, and running gags on food quickly become stale when the office only employs 12 people.
Regards,