Make sure the center of the weight in your rear bags is not behind the center of the rear hub. Otherwise, you could have trouble. Get the weight as far forward as you can without heel strike. Depending on your bike and the length of the chain stays and the size of your feet and the length of your crank arms and the size and shape of your bags, this may be anywhere from easy to impossible.

If you cannot get the center of the weight ahead of or on top of the rear hub, then I would definitely get front bags too.