It's usually not that bad, mostly afternoon showers but every several years, there's a bad one with some flooding. The dry season (our summer, their winter) is the best time to go. So your May/June timeline is good, I just wanted to remind you that it will be cold at night. It usually doesn't get much below freezing, this year is not normal. Besides, if it's raining, you have to worry about hypothermia and better to be cooler but dry than a little warmer and wet.
Other random thoughts. Travel as simply and as lightly as possible. Fancy bikes are great in the States and in touristy areas like Cuzco but if/when you get in the back country, you'll be doing your own repairs. Bikes are popular everywhere so even in small towns, you will be able to find 26 inch tires and tubes (if you shred one) but you won't find 29s. And they will be Schrader valves, not presta so know the difference. You can pedal clipped in if you want but I prefer toe clips and straps and comfortable light walking or hiking boots. I usually don't carry a spare tire and twice I shredded my rear tire over the years. The first time, I only had to walk 4 miles to the nearest town, the second time about 20 miles so after that I got into the habit of cycling in comfortable walking shoes. Plus, if you stash your bike in a hotel room for a day or two, you can go on a hike.