I've taken it out the way you tried, using a big flat-blade screwdriver. Or maybe the back of the blade of an old butter knife. Something flat, and big enough to fit into the slots in the "crown" without wiggling too much.