+1, The check valve isn't working. The most likely cause is rust. Pumping tends to "squeeze" water out of the air, and it accumulates at the bottom of the pump, eventually rusting either the ball, or seat of the check valve.

You need to disassemble the manifold between the base of the pump and the gauge, and clean or replace the ball (if that design) or otherwise service whatever check valve is there. It's usually not difficult and should take about 10 minutes or so.