You should be able to pop it out by hand. There is a very small ridge on the back of the plastic bushing that keeps it in the bearing bore. You can pop it out with a screwdriver blade, if finger pressure doesn't do it. There is a rubber lip seal on the bearing(gray seal with dirt under it in the picture) that is supposed to prevent dirt from getting into the bearing. Dirt will accumulate under the lip, as it is in my photo, and cause the crunching sound you hear. You can carefully lift that lip seal and wipe the dirt from under it.
Typically the non-drive side bearing will go bad before the drive side in a GXP bottom bracket, due to the design using that bearing to prevent lateral movement. If either bearing feels rough or has play, when turning by hand, the bottom bracket should to be replaced