I agree with others that if the chain is falling off to the outside, it's an adjustment problem. But I recently saw this dual-side chain catcher from Wickwerks that claims to prevent dropping from inside or outside.

I'd try to address the drop to the outside first with adjustment of the front derailer. I run a triple and use a chain catcher to protect from drops on the inside. I set it up 20 years ago, works great. Good luck!