I had similar difficulties with glasses-mounted mirrors. I then tried the
Mirrycle Road Mirror for STI and loved it from the very first ride. I have them on all my bikes now.

I have the old version that got recalled. Its good to see they came out with an improved version. I should call them and see if I can get the new one. I kept the recalled mirrir as it was SO much better than the Blackburn mirror. The Blackburn mirror vibrated horribly and also didn't stay put at all. I have the STI version on the road bike and mountain version on the hybird commuter. Excellent mirrors. I only use one on the left side... I don't get cars get to the right side of me if I can help it so I don't need a right side mirror. Being able to see the traffic behind you with a very quick glance is super helpful when riding in traffic. I can frequently see traffic before I can hear it.
Happy riding,
André