It's easier to set up your floor pump/frame pump for presta. Then you don't need those adapters. Even if you keep the adapters, you don't normally leave them on the wheel all the time.

With presta, the air pressure closes the valve (instead of a spring doing the job on a shrader valve) and you screw it in to secure it. Yes, when you screw it or unscrew it you can accidentally overcome the air pressure and manually open the valve. Just be careful to not do that. Pull gently on it as you screw/unscrew it. Having the adapter doubles your chances to let air out by mistake since you have to remove the adapter before you can screw the valve closed.