I know you can get maps with bike friendly roads marked in various countries, but does anyone know of websites that have such maps? I'm especially interested in France, Italy, Belgium, Netherlands.
Check the specific countries tourist information sites. Usually the (on line) maps are not good enough to cycle with, but will give general routing ideas when used with more detailed road map (eg michelin series). Some countries will offer better, detailed paper cycling maps, either free-or for a fee, at tourist kiosks or by mail.
I often use this site for my route planning.
Goto "Routenplaner von ViaMichelin" in the right center and select one starting place (Abfahrt) and one ending place (Ankunft). You can select accuracy, too.