You SHOULD be concerned. You can't get away with stuff on carbon parts that you can with alloy.

A shim CAN work but the key is that you want it to be a precise sort of thing and not just a bunch of wraps of pop can metal. The shim has to pinch the bars just right so that when tightened in the stem there is even pressure all around the circumference of the center section. You just can't assure that sort of accuracy with a generic shim or wrapping it with multiple turns of metal shim stock without some careful measuring and allowing for the inter layer slack generated in a multiple turn wrapping.