Some clothes are much easier to clean than others. Nylon running shorts, for example, make great swimming trunks; you can also ride in them; and they clean very easily (just going swimming, or showering with them does it). They also dry quickly. Wearing them for a short time does it.
Some clothes also seem to hold less dirt than others.