I took a picture of the stem with the extra spacer below, just to see what people thought since it is always better with a picture.

Would this have been a problem if I decided to leave it like this?

The top of the steering tube is flush with the bottom of the top screw. It seemed like it held fine, but I am not sure how things will handle over time and with real road conditions and speed.