You need a little space to accommodate using the spacers and if the tube reached the top of the stem, you might not be able to properly tighten the screw before it tops out.

Adjust the spacers until the bars are at the height that feels right for you. For example, if when you are climbing hard and your front wheel jumps up and off the pavement, lower the stem by removing spacers below it and place that same spacer over top of it.

In other words, you are good to go, barring any height adjustments.