Fear is natural. I have fallen while in a doorway. Front wheel popped off.
It gets better. Although I still dunno about rollers sometimes. My problem is that I still pedal so hard on them (to keep the wheels spinning for balance) that I burn myself out within 10 minutes.
It is an amazing feeling once you get them. And honestly, the worst fall isn't any worse than in the video above. You just kinda tip over.
-D