if what mathepat said all checks out, then it could be your front rim is bent (doubt it, or you would probably know and if you have rim brakes, they would rub,more then likely.
I'm going to go back to my first thought and say its the geometry of the bike, have someone else ride it no handed and see if they get the same thing....?
if it is the geometry, then you simply need to practice riding with no hands more.
