This is part of the route I toured in 1989 which started out in Luxembourg. Watch out for Brugge. You'll never want to leave! When we left Brugge, we cycled to Holland. Took the Ferry to Zeeland. Don't miss riding along the route where the Deltaworks are. Impressive. Ridden a lot in Luxembourg. Small, rolling terrain and lovely. E-Mail me directly if you'd like some more info. It's been a while, but I'll be happy to convey what I know. The photo attatched is in Luxembourg.
I recommend going West. If you can, I also recommend the North Sea coast south of Haarlem, but that is out of your way. Basically, head to Brugge - one of my favourite cities in Europe - then go to Gent, then to Brussels. From there you can keep heading southeast to Luxembourg. You could detour south through France, but I loved Belgium so much that I did not want to myself. Lux. is great for cycling, but a little expensive. Lots of major cycling routes. I just finished that very route myself last month. If you have any questions, PM or email me.
I spent about a week on my bike in Belgium last fall. Another vote for spending time in Brugge. The real gem is not so much Brugge as Damme, which is maybe 10-15 miles out. Damme was unquestionably the highlight of my trip.
I'd suggest you go from Brugge to Ghent (~35 miles); spend about 1 day in Ghent (it really deserves more, but c'est la vie). Go from Ghent to Mons to hop on the RAVEL network.
I'm afraid I can't second ggy's recommendation to bike to Brussels, as the area around Brussels is a bit industrial and IIRC does not have a lot of bike routes. The bike paths really aren't made as much for intercity travel as they are for recreational use; most Belgians either drive or take the train to go from one city to the next.
Flanders is as flat as a board, so if you want hills it's off to Wallonia and the Ardennes. The "RAVEL 2" route along the Meuse is pretty sweet -- amazing scenery and houses just south of Namur -- but I'm afraid to say that Dinant itself is rather unexciting. Namur to Dinant is about 30 miles btw.
One minor thing to watch for is the occasional rough road conditions. The vast majority of the time you'll either be on well-paved multiple-use paths or on generous shoulders, which is fantastic. However, many of the older cities still have cobblestones (not fun) and some of the RAVEL routes have gravel stretches.
Last but not least, make sure you get some good maps! The RAVEL network and the Fietsroutes have some markings, but signage does drop off (usually when you need it the most ) and the RAVEL routes don't extend very far into the south of Belgium. Fortunately, most bookstores in the big towns will have a good selection of bike maps.
While Brugge is very much worth visiting, I found the cycling better the further east I went in Belgium and northern France. You said you like hills, and you'll also find more hills further east. The west is largely Flemish, and as someone wrote above, Flanders is very flat. It sounds like you have a reason to go to Reims, but I found the cycling between Reims and the Belgian border pretty dull--one of the few dull spots in France. Luxembourg was quite pretty and quite hilly.