That helps a lot. But you have it backward. The chain runs from the upper pulley to the sprocket, and should therefore be falling off the sprocket. That's how derailleur shifting work.

If it's happening when pedaling forward, odds are you have a badly worn or chipped upper pulley. If it's happening when pedaling backward, it could be the same, or simply poor trim (derailleur pulley not centered under the sprocket) which will also make noise, and possibly cause an unwanted shift when you don't expect it.

Resolving trim issues depends on whether you have index shifting or not. On an index system you use the barrel adjuster to adjust cable length and center the RD for good shifting and quiet running. Without index it's more a question of developing a bit of skill and listening to the RD for the sounds of poor trim.

BTW- if the problem is happening when backpedaling, there's another common cause. As you backpedal the unguided chain will often shift one or two sprockets over. You may not notice this happening at the instant it does, but as you continue the chain is now misaligned with the upper RD pulley and, since it's running backward, the sprocket is in charge and will derail it off the pulley.