In exactly the same way as you cannot push a car forward by pushing at the steering wheels, you cannot push a bicycle forward by pushing at the handle-bars. Period. All notions to the contrary are erroneous.
No, the 'conservation of momentum' does not apply to the push of the cyclist upon the bicycle. This is because both exist in the same inertial frame. Think of being in free fall with a ball, a bowling ball, you must push at it with more force than that which is required to move the ball a free fall speed. You would need the assistance of an extra-terminal-velocity force to push the ball any fast downward than it is already being pulled.
Now if you had a ball while riding on your bicycle, you can apply enough throwing force to hurl it outside your bicycle's mph inertial frame. But, without that much force, if you simply toss it straight up, it travels forward inside the inertial frame of the cycle. And this is what's happening when you push forward at the handle-bars, the force is entirely existent within the cycle's inertial frame and is in no way sufficient enough to move the bicycle forward anymore quickly than the bicycle is already moving.
To do this, you'd have to get a good push from behind. If you were moving at 30mph, you'd need an auto to meet your inertial frame by bumping your rear wheel at a speed greater than 30mph. Then you'd experience a jerk forward...note than when you push at the handle bars in a race, no forward jerk exists--the cycle doesn't break momentum.
Also, one cannot pull at the handlebars and slow a bicycle. All than can happen is that the front wheel lifts off the ground, but the velocity will not slow. This is like checking your math, if the reciprocal operation is impossible, then the initial operation couldn't be possible.