I went through part of Umbria when cycling around Tuscany a few years ago, I went from Spoleto to Norcia, across the Plano Grande and onward to Assisi. My memory is of very scenic, but very rugged countryside, and the best cycling roads tended to be narrow and steep. My advice is to put the lowest gears you can on your bikes, travel as light as you can, and accept the climbs as the trade off for the beautiful scenery.
Some of the main roads aren't nearly as steep, but they are more likely to have tunnels (that's how they avoid being steep). Get a good map that shows the tunnels and bring lights if you plan on riding these roads.
The Touring Club Italiano (TCI) regional maps, usually in a 1:200,000 scale, are excellent cycle touring maps. They show lots of minor roads, and have relief shading to give you an idea of how steep the terrain is. They also mark the steeper climbs/descents. Get them from
www.trektools.com .