Yes, you can use your setup as-is -- adding spacers would be done for cosmetic reasons only. However, I agree with Metzinger's suggestion to flip the shim, install a 5mm spacer and put your stem on top. This would both provide a proper bearing surface against your headset, and would be visually appealing. As several other posters have mentioned, you can go into any bike shop and pick up a 5mm shim, they might charge you a dollar or they might just give it to you, depends on the shop and whether they know you.