I have learned that in life there these small things that get you inspired, and if by wearing the yellow jersey someone out there gets their kick and that little extra motivation, then I do not see what is wrong with it.

But it's another matter for someone to wear it and act like a poser. As long as the jersey doesn't go to the rider's head, it is perfectly ok.