What scroca said plus pick clothes that dry quickly and are comfy when damp. That goes for jerseys, shorts, and socks too. I find that some synthetic socks dry very quickly and are reasonably comfy when damp any way. Others are not so good that way, find what works for you. I like the low cut poly Under Armor socks that come 4 pairs for about $13, but I don't know if they are available where you are. Shoes that don't soak up much water and have plenty of mesh are ideal since they allow socks to dry while you are wearing them.

I avoid cotton at all costs and am not crazy about wool either, but a lot of riders swear by wool.

Drying clothes when camping isn't usually much of an issue when camping. When staying in a room it is even less so.

Three sets of clothes seems excessive. I have two sets of on bike clothes and one for off bike, but can get by with one set of bike clothes if I have to.

Edit: I forgot to mention... In the rain I put damp clothes under a pannier flap to dry.