Well my problem with them is simple enough. On an ordinary bike with ordinary handlebars, anyone can just put the saddle at the right height and ride, right? It's easy.
But on a drop handlebar bike, it's not easy. People talk constantly about finding the right "fit". What is fit? it's the precise location of the handlebars so that you don't do yourself a mischief. How do you determine correct fit? If you're a dentist you pay someone to take your measurements and strap you to a contraption and then give you a printout. If you're not, you ride the bike for ten seconds, hurt your neck, make an adjustment, ride the bike for ten more seconds, hurt your lower back, adjust again, hurt between your shoulders, over and over again until you find the least bad position. If you're particularly lucky, you'll eventually find a position that's still unpleasant but tolerable. If you're not, you'll buy several stems on ebay and repeat the process with each of them.
Then you still have to ride every day for a couple of months so that you can build up core strength, which will let you support your torso in this horrible leaned over position without having to put weight on your arms. Then it finally approaches comfortable.
Some people may be lucky in that their body is already fit enough to tolerate the position, and they might be doubly lucky and find the right "fit" first try. Those people are witches.