I have 3 road bikes and a CX bike used on the road at various sizes. One 54 cm, 2 55 CMs, and a 56 CM. All are setup for the same saddle height, saddle to BB offset, saddle to handlebar reach, etc. With the varied geometry of each frame I've had to use quite the variety of stems. I'm even running a short 70 or 75 on the one bike. I really don't notice a difference in terms of handling for the most part. If I've been riding one bike primarily and then take out another I may notice a difference for the first 5 minutes or so and then never notice it again for the remainder of the ride no matter how hard I am going. A bike's handling is affected by so many factors and I believe stem length is just one of the many variables and by itself is not that big of a deal. Just my two cents, take it for what it's worth.