If you are male and ride in traffic, you do not want stem shifters. Should you hit a stopped car or opening car door and you have a lever pulled back, you could impale yourself on them and find yourself castrated. (At a race many years ago, I has a bystander come up and tell me that happened to him.)
For down tube shifters, taking your hand off the bar to shift means no accelerating while your hand is off, hence a slower start. Now I raced back in the day when that was all there was. We dropped a couple of cogs while rolling up to a light and did one shift midway through our acceleration. I now rider fix gear a lot. No shifts. I don't get dusted by guys doing multiple shifts. (I do get dusted by guys with quads I was never given and guys 40 years younger than me but all the gears in the world won't help those issues.)
Ben