most obvious culprit would be loose screws on your cleats. I took off on my bike with my second pair of shoes one day after not wearing them for several months. I rather quickly remembered that I had swapped cleats around and not tightened this set down completely.

Other than that, we'd need to know what type of pedal/cleat you've got.

1) Old worn cleats
2) Worn down pedal interfaces
3) Play/Slop in the pedal spindle/races
4) ????