There actually is a definite subset of riders, mostly newer, but some not, that wobble all over the road when they're riding, even when the road is dead flat and clear. I've seen a fair number of these folks on the noncompetitive group rides I used to do in the area years ago, and always wondered why they couldn't hold a straight line.

However, having used rollers a fair extent in the past, I found it really helped my smoothness. Smoothness of pedalstroke in particular. That's useful for accelerating safely with a group.

The balance thing though does become noticeable when you start practicing the useful move of 'looking back to check for cars' on the rollers. You might think you hold a dead straight line when you do this outdoors, but if you actually try it on the rollers, you will likely be horrified at how not a straight line you hold once you look back.

If you're good at that move, you will be able to ride the rollers continuously even looking straight backwards, like over a minute.