If you use a low rider rack up front with up to 30 pounds in panniers it actually makes the bike more stable because of the low center of gravity of the load. Bar bags have the opposite effect.

The key is learning how to distribute the weight between front and rear and equal on both sides to prevent shimmy.