Get an exacto keyhole saw. Or, if you can't easily find that, a knife with a blade edge that is parallel to the axis of the knife handle (like the pic below). Take that knife and place its sharp edge against the face of a sharp file. Whack it with a hammer. The file should cut little "teeth" in the blade. But a keyhole saw is better.

Saw several radial cuts in the remaining plastic cap, almost to the crank arm threads. Then use a chisel to break off part of the cap. The rest of the cap should then easily be removed using an awl, or a dental pic.