Like many I came to MTB from road, even did some road and track racing. So I initially treated mountain biking like road biking in terms of clothes and bike setup.
I found other people on the trails to be a lot more talkative and friendly when I'm in normal clothes (non-baggy hiking shorts an t-shirt) vs. my collection of roadie jersey's and bibs/shorts.
I also find I'm out of the saddle enough to not really need any chamois and crash enough that I don't want to ruin any of my jersey's (any more than they already are anyway...).