I believe there is a check valve in the base of the pump where the hose attaches. Sounds like it is dirty or stuck open.
__________________
2010 Cannondale CAAD9-4, 1978 Jack Taylor "Tour of Britain" #7338, 1969 Jack Taylor Ladies "Tourist" #5935 (wife's), 2010 Specialized Dolce (wife's)