Whether or not stiffness matters for efficiency, it can make a difference in handling and tracking. This is a big factor in mountain bikes, but I think it is also true to some extent for road bikes.
Regarding the effects on pedaling, in particular out-of-saddle efforts....
I don’t think anyone can deny that out of the saddle hammering on a stiff vs flexible frame feels different. So clearly something is different. Until you can fully explain WHY it feels different and give an explanation as to why that does not in fact matter, then you cannot categorically assert that stiffness does not matter.