Grab some chain tensioners at the lbs and call it a day. They should be able to take care of your problem so you will not have to worry about it anymore. Maybe the "crash" affected the dropouts in such a way that the nuts refuse to "bite" into them properly anymore. Serrated washers might also help but they will seriously tear up the paint on the dropout.
-Ryan