Hahaha turns out it was my fresh paint job making it seem like the thing was cemented to the tube.
I pried it out in 30 seconds flat. The shim is in great shape too so I'm gonna hold onto it.
Any idea why shims are used on seat posts to begin with?
My best guess would be the 22mm post is slightly lighter?