Just wondering if you are using a conventional saddle. Your experience reminds me of the time when I bought one of those split saddles with one articulating pad supporting each butt cheek and no saddle "nose". With the nose of the saddle absent, I found the one-handed handling to be quite squirrely, even for short times needed to reach the down tube shifters. Fortunately, the saddle had a satisfaction guarantee, that provided for money back if not satisfied in any way, of which I took advantage. Until you try it, you don't realize how much the nose (or horn) of the saddle adds to your overall stability on the bike.

If this is not the case with your bike, then I would agree with DnvrFox that the "trail" is probably the culprit. I suspect that the model of bike that you bought is a typical racing geometry, which is characterized by responsive steering (also sometimes reffered to as "twitchy"). Touring bikes typically have a longer wheelbase and longer trail which provide a more stable ride. If this is the case, my advice would be to keep on riding your bike and you will soon get used to the responsive handling.

With a smallish frame such as yours and short trail steering, it is very common to encounter toe overlap, a situation where your toe can contact the front wheel when making an extreme turn, such as a tight, low speed U-turn. If this is the case, exercise caution when making U-turns, especially if you use clipless pedals or toe clips. Good luck.