Thank for all the help guys, I took the whole thing apart and reassembled it to be sure, everything is tight.

It is definatley not just the cranks because if I am just seated and I put pressure on one side I can see the other move.

I never even considered that it could be the frame flexing, this whole problem is occuring on an aluminum frame I just built, but when I am looking down watching the flex the bb doesnt appear to be moving at all