Regnak, if you're not holding onto the stem or bar, of course its going to go left or right. I don't have the problem you are describing and my wheels are still out of true. Need to get them trued at the LBS.