Well,
there would also be loss due to more friction of the tire against the road. The less tire against the road, the more efficient you will be.
As someone who weights 152 points, I just cannot flex a frame much to notice on any of the frames I have ridden. I think a lot of these individuals who discuss how certain bikes take off like rockets are ignoring the different components (wheel set etc..) between their lower-end bikes and these rockets they ride.
Maybe there is the feeling of immediate transfer from the pedals to the wheels, and my CAAD frame does feel different from my steel Fuji. But how much of this is weight, wheels and the crank-set being used? Unfortunately there are a lot of anecdotal descriptions of stiff vs. noodly frames, but I still have not seen a lot of evidence statistically as to how much flex really impacts riding efficiency.