Originally Posted by FBinNY

Before going into details, consider that your bike has a derailleur to guide the chain onto sprockets when pedaling forward, but no guides when backedaling.

Without any guides the chain can derail (especially in the back) when feeding from an angle. The teeth catch an inner plate lifting the chain and starting the process of moving to a better aligned sprocket. If you have a modern cassette with cut down teeth (shift gates), then this type of derailing will be more likely.

In most cases the chain won't derail from middle sprockets, but is more likely to as you move to the inner ones, especially if it's still on the larger (outer) chainring.

Cleaning and oiling the chain may improve things slightly, but there's not much you can do otherwise except to remember not to backpedal in the gear combinations prone to derailing.