On a stand the rollers indeed work well ensuring accurate shifting. Unfortunately, the real life situation is dynamic: the chain pushes against the pulley, the tensioner arms flex, chain jumps around.
The tensioner arm may flex a bit but the shifting pulley axle cannot flex that much, unless one of bolt is loose.