I'm far from fit as a rider right now, only been riding for a couple of months after 5 years off the bike. But I'll bet I can drop 99 out of 100 riders who ride with tennis shoes. Most rather quickly and most of the rest after a few miles at pace when the strain of having your foot flexed on each power stroke.
Good cycling shoes are much the same as good hiking boots. They are of little use for a walk in the local park, but go on a real trail with a pack on your back and they make a big difference. Also like hiking boots they often take some getting used to.
When I first went clipless over 15 years ago I only had one fall, the very first time using them. No clipping out related falls since then. They simply are not all that difficult to use. Perhaps I had an advantage as the clipping out is not unlike clipping out of downhill skiis.
15 years ago there were some good reasons to not go clipless. Systems where you did not have to walk like a duck were just getting started. But today there are several well established systems that have shoes that are quite nice for walking.
This is not to say that one should go clipless for every person for every bike. Just as it would be foolish to put $200 tires on a junker car worth only $150 it can be foolish to put a $150 pedal system on a $100 K-mart bike.