Try tapping down on the loose bolt.

When installing a plug, the tightening of the bolt pulls up a wedge inside to hold it in place. When you loosened the bolt, it didn't necessarily cause the wedge to slide back down. Tapping on the loosened bolt should release it.