You can also roll your bars a bit to change the position of the hoods to raise you up or down a little (while riding on the hoods).

So you could flip your current stem, and if the change is too much, you could roll the bars back a bit.

Or just reduce spacers. Some combination of those three adjustments will work for you.

In my opinion: better to be a bit higher and be able to ride comfortably in the drops than to be too low so you don't want to ride the drops, or can't breathe effectively when you are. You are more aero in the drops on higher bars than on the hoods with lower bars.