I'd try crazy glue (cyanoacrylate) without doing anything except cleaning the two surfaces. The (epoxy?) glue on the shifter looks solid, and cyanoacrylate works well on smooth surfaces that fit together tightly.
But if it doesn't hold, then you'd have to deal with the residue glue on the frame. (Or maybe the glue would all be on the shifter, same as now.)