Practice Practice Practice! I find even switching between bikes time to time I don't feel so comfortable riding hands free !

After doing about 1000km on my road bike I finally found that I can actually RIDE it and steer it for a good few hundred feet at a time with some slight curves as long as the road/path isn't to choppy !

I don't recommend filling your hands up with stuff while doing this either, you should always be alert and ready to grab the bars !