Well if your frame is flexing, then that's lost efficiency - some of the energy you put into pedaling is being lost to frame flex.

The amount of speed lost to frame flex is probably somewhat negligible - but it'd be an interesting experiment.