Bah, who cares.
Everyone at work knows I'm not a drunk, poor, or incompetant....they just accepted me as the kindly weirdo on a bike
There's a difference between doing something because there's no other option, and doing it because it's your passion. If your co-workers can't tell the difference, f them, because they obviously don't seem to know you enough to realize this difference.
Oh, and the painter was using an Xtracycle....those are not cheap. So he was probably an entusiast as well. It's not what you ride, or how you ride....it's that you ride in the first place.