Might be they see you as too... how do i put this.. too much of a presence on the road. Whereas everything else is more subtle so you don't stand "out" in the road, so to speak. Plus it also makes you look slower.
I'd say wearing a jersey like this would probably get more attention without looking like a construction worker.
