Frame bags will make you more aerodynamic in a headwind, but can cause issues in a cross-wind. I'd be much more inclined to use one for offroad situations.
Lifting a bike may be easier with a higher center of gravity, but it will also result in slightly less stable handling. For most long distance events it's not really an issue, unless you like to carry a lot of gear, are marshaling a ride etc. But for heavier loads, I don't see the point of sacrificing ride quality for lifting, unless you plan to do cyclocross races in mid-tour (or are going off-road, and need to lift the bike a lot).