I have been struggling with the same issue with my trekking bars. I have the same mirror and it does seem to need a lot adjustment.

I have been thinking of something like this (not this one -- something better) that would allow the mirror to be below and to the left of the bars. It would need a longer extension to work the way I am hoping.