That's the way I felt after a professional fitter "fixed" my position by flipping my stem upward. It actually felt a bit better that way on the trainer, but when I rode all the way home on the drops I knew my torso wanted to be lower. I ended up switching to a bar and stem that raised my hands about a centimeter from where they were before the fitting.

My rule of thumb is go as low as you can or you want, as long as you can see and breathe, and control the bike.