I used to carry a spare pair of shorts and washed one out in a sink or the shower when convenient. On more recent trips I have taken to only having one pair of bike shorts. I wash or rinse them out when convenient, but if I go a few days between washing or rinsing it isn't the end of the world. If they don't get washed I sometimes turn them inside out and let them get some sun. That kills whatever might grow there about as well as a good washing. They may be damp when I put them on some mornings, but that really isn't a big deal since they get wet with sweat quick enough any way. I have gone with one pair on a few long tours now including a coast to coast one and have not found it to be a problem.

The bottom line is that two pairs is more than enough and you could get by with one if you wanted/needed to.