Oooh! Stiffness! Perhaps better called Rigidity--Stiffness is a property of a material. What's being discussed here is rigidity of a structure.

I skimmed a bit and saw the Merckx hour record bike mentioned--I have touched that thing and it is crazy light, especially for the era. It was, however, probably a fairly close match for what he was riding on the road as far as rigidity is concerned and that was probably an important consideration.

My own personal experience is as a trackie who favored sprints. A rigid frame is nice to have when you're out of the saddle because you can feel confident in where the wheels are going. I had this in my track bikes and in a crit bike I had, with slightly oversize steel tubes on that crit bike and one of the track bikes.

My main road bike, to be contrary, was a Vitus 979. So flexible that for the first year I had it, I was convinced I had a tire going flat. And at no time going down the road did I ever think the wheels were in the same plane. But for anything but the last 200 meters it was just fine. More than fine, as it wasn't beating me up.