I reccomend you put a chair on one side to mount the bike and a wall on the other side to lean on.

Once you get pedalling, keep looking forward and just keep riding. Staring down at the rollers will make you panic. You say the rollers want to guide you to the side, but that is an illusion - the rollers guide you nowhere and it is YOU who are guiding yourself to the side.