We use the shoulder strap if we are carrying them far, boarding a train, etc. We often use the handles to tote the bags around when going shorter distances, like carrying around a campsite or into a hotel.

I do carry an extra insert in my parts bag, but have never lost one. I've thought about putting a drop or two of Crazy Glue on each insert to make them more permanent, but sometimes I have to swap out the inserts for different size ones since we have different racks on our bikes (Tubus vs. Blackburn), which have different tubing sizes. If I was using the bags on a single bike 100% of the time, I'd surely glue them in for extra safekeeping.