That would make sense. Some or all of what you pump in comes right back out. It's not a pump anymore but a see saw, lol.
I assume the valve is in the brass assembly where the gauge and hose attach. I have never taken that off of mine. I guess you would treat the fitting like a gas fitting. I am not sure if there is a compound that you should apply to the threads when you reassemble (maybe just plain teflon tape will do).
Where did you find the replacement valve? Let us know how the replacement goes.