If it's not cold outside, I don't mind the rain too much if it starts to rain while I'm riding. If it's already raining out, I'll rarely head out into it. A light rain can often make the surroundings more beautiful, so there are upsides. What bothers me are all the headaches from riding in wet weather:

1) I don't like twisty descents on wet roads, and I don't like descending with poor vision from wet and foggy sunglasses and/or without sunglasses if I take them off.

2) If it's cool and it starts to rain, then my clothing gets wet, heavy, and dirty. No fun, IMO.

3) The bike tends to get real dirty on a rainy ride, so I always hose it off gently, wipe it down, and then relube the chain and other moving parts as necessary. Not a big deal, but not something I enjoy doing.