Try using a small drift punch [perhaps 3/32"] and tap the broken piece of allen wrench on its edge from different angles. This should 'un-wedge' it and it will fall out. Be careful not to hit the hex area of the bolt itself. If you swage the hex hole you might permanently lock the broken piece in requiring more extreme methods of removal.

Just for future reference . . . the 'ball' end of those allen wrenches are not for high torque [as you discovered]. Loosen the bolt with the non-ball end first . . . then use the ball end for quick removal.