It only happens to me when I'm going slow enough to turn the wheel without leaning into the turn, like if I'm messing around doing no handed no footed upside down rainbow trackstands or something super amazingly cool like that.