A little rule of thumb is that top of stem must be about 1/8” (1/4” max) higher than top of steerer and that’s what limits the amount of spacers under. Anything less is risking your safety.

Do you care about aesthetics? High sloped stems make your bike look dorky but it may be the only way to go.