Keep riding is all I can say.

When driving, it helps to look at stuff on up ahead of you, not the ground right in front of you. Maybe this would make a difference.

Some bikes have more stable steering than others, relates to how far forward the forks sweep and stuff. It can make it easier or harder to ride without hands. If you can't ride without hands, it likely isn't you. If the steering seems twitchy, that may be part of the reason, too. This can happen with cars as well. A little bit of toe-in in the front wheel alignment tends to make them go straight by themselves.

If you have a chance, ride some other people's bicycles and see how they are different- that might show if the problem is the bike or if it's you.

How do you walk? You've probably done it most of your life, but you'd have a heck of a time trying to tell someone how to do it over the internet- it's mostly just second nature and you don't think about it.