Originally Posted by
jland
I would be tempted to put a spacer under the stem to try to get it flush with the top of the quill adapter. I have a setup like this and like it a lot - especially the compact bars. My adapter came from ebay and doesn't have the flange under the top section.

It's more than for looks. The main reason for having a spacer above the stem is to assure a threadless
carbon steerer is fully supported by the stem clamp. For your aluminum adapter, that is unnecessary and you don't even need a gap below the stem to allow headset adjustment so the stem and adapter top can be set flush with each other.
Removing the top spacer and placing it below the stem will allow you to lower the adapter by the same distance (10 mm?) while keeping the bars at the same height. No reason to have the adapter sticking up any further than necessary.