Bought a good water bottle cage carrier combo over the weekend. I haven't tried it with a full water bottle yet, but the setup does not wiggle/rotate.
This is a Giant brand handlebar cage clip. $6 from LBS.
To that, I have mounted a Novara brand water bottle cage that matches my raw lacquer quite well! I forget, but I think it was $12 from the store that sells the Novara brand (in case I'm not allowed to name it in the forums).