I had that problem, too, but then found out that if the recesses for the bolts aren't facing the backside of the bolts they can wriggle out pretty easily. Just flip it around and the problem is solved.
Originally Posted by
Zemo
I had a similar problem after putting on a new chainring. Turns out the chainring bolts were coming a bit loose after riding, allowing the chainring to move on the cranks a bit. Make sure those suckers are tightened down hella, and make sure your chain is tight enough.