It's really hard to lose muscle-size and mass once you've built them up. You may lose a little, like 10% with not working them, but can take decades to lose more, and that's without working out even. After 10-years of not riding, not lifting weights, not running, not doing anything but working in an office and watching TV, my muscles have lost 75% of their strength, yet my 15" calves and 24" thighs are only 1" smaller than at my most fit phase.
The only thing I can think of to slim them down is to lose weight. Lose as much fat as you can by during endurance rides. The lower weight will tax your muscles less for the same amount of effort pushing your body around. Also spin easy gears and never strain your muscles. Stay above 90-95rpms on the flats and above 80rpm on the hill. By using primarily slow-twitch aerobic muscles, you recruit more of them into use and lower the usage of the fast-twitch anerobic muscle fibre. This in itself won't reduce the size of the muscles by much, but it'll improve your riding speed and endurace.