Lots of responses, but I didn't see anyone talking about framebags specifically, so I'll chime in here:
The bike's center of gravity is higher than the top tube, naturally, because your weight contributes to most of the overall weight of your touring rig (in most cases). So, a framebag is still below the "balance point" so to speak. I find that the further weight is from the center
line of the frame, the more it contributes to sway and unbalance.
I have never found a frame bag or an under-seat bag to contribute greatly to sway. If I really overloaded my seat bag (Revelate Viscacha), it could sway and contribute to unbalance, so I swapped it with somebody for the slightly smaller Revelate Pika and stopped loading it to maximum capacity. Problem solved.
As far as a framebag goes... It will only contribute to your balance. It won't detract from it at all.
Two other suggestions:
1) Get a full-size framebag made (or pre-made) and keep your tools in the bottom of it, right next to your BB. This counterweight improves handling a lot.
2) Bedrock Bags makes a pretty neat bottle holder called a Honnaker Bag:
Honaker Hydro Pack ? Bedrock Bags and Packs If you keep this bottle partially full as a backup water source, it'd keep your center of gravity real, real low and balance the bike quite well.