Cable needs to be tighter. If you pull the cable away from the downtube with your hand, you should see how the whole system works, cable causes the RD to move inboard, releasing cable lets RD spring move the RD outboard. Especially if chain is off all becomes clear.

Gotta get that slack out so that first movement of shifter pulls the RD over one gear.