I wish I knew the answer to the "balance" question.

I suspect there isn't a clear-cut one. Yes, the body automatically and with ease compensates for the empty side of the bike, but are stresses still set up in the frame that could eventually lead to weakening it? I doubt that the frame "knows" about the compensation, since the compensation is dynamic but the weight on the frame from the one pannier is static.

Apparently long-distance tourists are very concerned to get weight distributed just so on their bikes to prevent shimmy and frame stresses over the long haul.

Oh, well, just something else to fret over.

P. S. Re-reading this before posting (how unusually diligent I am this morning!), it sounds like something MichaelW might have some good thoughts on.