You don't really elaborate on what "a bit dicey" feels like so it's hard to help with a definative answer.

Could be a couple of things. Fit and position have been covered by others. Experience is another component, I know when I came back to the road from riding MTBs in the dirt, I found the much quicker steering to be a real shock. My handling has improved tremendously over the last couple of years.

One thing to try - can you ride the bike without hands at all? That skill is partly you and your balance and the set-up on the bike. Every bike should be hands-free capable if your inner ear is working properly and the bike is set up correctly. If the bike veers, one of those things is amiss.

From a mechanical standpoint, I'd check two things - your front wheel skewer and your headset.

Are they both tight?

Check the headset by applying the front brake and rocking the bike back and forth with your other hand on the top cups of the headset. There should be no movement whatsoever. A couple of times I've had one go loose and the handling went to hell.

Skewer is obvious - make sure it's tight and the wheel is properly centered.