I biked in Burgundy about a year ago, including the stretch from Noyers to Auxerre. I think that stretch would be pretty good for family touring as long as you pick the minor roads, which isn't difficult to do. Noyers is a gorgeous medieval village. I stayed in a very nice B&B there. The roads were wonderfully quiet between Noyers and Chablis. There was some traffic the last few kilometers before I got to Auxerre, which has some very pretty old houses in the center of town.
Have you looked at any of the bike trails in Burgundy? There is an excellent network of them and they'd be ideal for you. I rode from south to north. I began in Macon and followed the wonderful voie verte
northward. It's paved and quite pretty. The only downside was for part of the way, it's very close to the main TGV line from Paris to Lyon and points south. The trains go by often and make lots of noise (rather suddenly) when you're not inside the train. Cyclists get to bike through the wonderful Tunnel du Bois Clair
. In Chalon-sur-Saone, you can continue northward on a trail to Chagny. And from there, north on a signposted suggested bike route on quiet farm roads (not flat, however) through some of the most famous names in Burgundy, such as Meursault, Beaune, Nuits St-Georges, to Dijon.
From Dijon, there's an unpaved canal trail which goes northwest. As of last year, you could ride as far as Ancy-le-Franc (nice chateau). There are plans to extend the trail, I believe. The path surface was sometimes a bit soft but quite pleasant. The ride from Ancy-le-Franc to Noyers on roads was wonderful, through rolling wheat fields and almost no traffic.
Information on bike trails and routes in Burgundy: