This is advice for everybody:
When you feel your crank getting loose, stop riding immediately.
Usually, the first indication of your crank arm coming loose is that the pedal starts to squirm as you pedal.
If your crank arm is loose, it only takes a couple of revolutions to ruin the crank arm.
Someone mentioned using strips of beer can for a shim. I found that if your damage is so great that you can use a beer can for a shim, the crank arm is shot. Get a new one.
I HAVE used stips of aluminum foil for shims on stripped cranks. This works pretty well, but it is probably due to the fact that the damage was minimal in the first place.