My conversion project started out as "hybrid to cyclocross", then became "mountain bike parts to cyclocross frame". Fun stuff!

Do the handlebars first and see if you like that riding position before you invest in brifters etc. This would be a good time to get a quill to threadless adapter for the stem. I'm sure you will want to experiment with length and rise,and the threadless makes it very easy to swap stems and bars.
I think your main problem will be tire selection. Loose gravel usually demands mountain bike tires.