Crank flex is not an absurd thing to occur. On one of my fixed gears, I had 3-4mm of clearance between the end of the crank arm and the drive side chainstay. At some point, I flexed it enough to knock the paint off in that spot. It was a higher-end crankset on a very stiff aluminum frame, but I'm a fairly strong and heavy rider.