I think wearing a team jersey is ok..These are your friends..Nothing wrong with feeling a part of a group you chose to join. As long as the Team colors are cool.
I lost out when Bike Forums did not choose my original choice- BF jersey. But, I still wear it proudly. Be great if we were to ever have a local BF ride and could meet our local compatriots...