It sounds like the handlebars are too high, and too far away. Without adequate weight on the front wheel the bike will always feel squirrelly and nervous, and the front wheel will wander around as you try to climb a hill. Try and figure out how you can distribute more weight to the front wheel by putting a shorter, lower stem on, or by using swept-back U shaped bars tilted down towards the back. Maybe you can find parts and help from your local community bicycle cooperative, or maybe they would allow you to swap this bike for a used smaller bike that fits you better. If there's a community bicycle cooperative where you live they'll gladly help you out for free, that's what they're there for.