At slow speeds, you just steer a bicycle like a motorcycle. As your speed increases a phenomenon called gyroscopic precession takes over. When it does, if you push on the left side of the handle bar the bike will turn left, if you push on the right side, you will turn right. You can easily test by having a long straight road, gain some speed and gently push on one side of the handlebar or the other. You'll feel the bike start to lean, just quickly stop or you'll turn. Reading about the physics behind gyroscopic precession can make you a bit nutty but here is a general definition.

Gyroscopic precession is a phenomenon occurring in rotating bodies in which an applied force is manifested 90 degrees later in the direction of rotation from where the force was applied.