You just have to adjust quickly. It's not unlike riding drop bars on the tops or in the drops, where you can't immediately reach the brakes. If you're riding in an area where you think you'll need access to the brake levers, such as in an urban environment or where you don't have much warning of wildlife crossing your path, then riding in a position where you can't reach the brakes in a snap may not be the best idea. On the other hand, there are lots of situations where you don't necessarily need ready access to the brakes and you can ride in a position that puts your hands further from the levers.

Another solution is "hooked" brake levers, similar to these: https://www.amazon.com/22-24mm-Brake.../dp/B08NPGN52K

You can configure those so that you can pull the main part of the lever from the main grips, or the hook or curved part of the lever from bar ends.