I'll wear lycra when I do XC. When I'm fooling around on stunts and drops, I have a pair of baggy MTB shorts w/ a liner, just for some abraision protection. They're a fairly heavy denier nylon.
And not only for abrasion protection, but I'd hate to tear a pair of expensive lycra out on the trails. But I don't care what people think I look like, I go practical, 100%.