Yes, the smaller lever can be taken off. You'll have to go about it from the back side. Unscrew the nut with the 4 slots in it, then you can pull off the bracket that holds the brack cable, and you will be able to pull off the small paddle.
The 2mm allen key may be rusted in there. Ive broken a few allen keys myself. Try to get the tightest fitting allen key in there, otherwise you may strip the hex shape.