I stay with the bike. I suspect most do. I don't think the phenomena you are noticing is restricted to tall riders. 20" wheels are inherently more unstable than the much larger wheels of standard bikes. The riders center of gravity is well outside of the wheel diameter and therfore the moment arm of any disturbing forces is very long. To hang off into a turn means unweighting the seat and in that instant an already unstable system is going to be upset further just at a time when you want to be setting up for proper corner entry speed, lean angle, etc.
H