I'd guess it's not actually a shim, rather someone was using a couple of 20mm spacers as shims.

Shouldn't be too hard to remove. Back out the bolts all the way and try to push it out. If needed gently pry the slot apart to see if it'll drop out of there.