Your steerer tube is a given length , then you have some spacers and the stem around it.

The screw at the top of the steerer tube is your headset adjustment.

the stem when the bolts at the back are tightened keeps the headset adjustment.

swapping spacers from below the stem to above it will be fine..
to lower your stem and bar height . flipping the stem over at the same time..
but to do that you loosen the headset adjustment, so have to do that again..
when you are done with the stem and spacers, being re arraigned.