If the cranks are loose enough on the spindle to move around, you are in trouble.
But, it is almost certainly frame flex. I ride several bikes regularly, and it is always surprising to me how much difference there is in the way one flexes at the BB compared to another.
By and large, if the frame is aluminum, there will be minimal flex there. If the frame is made of good steel tubing, there will be little flex there. If the frame is made of hi-ten or low-end Cro-Mo, then there can be a ton of flex there.
jim