If you were riding it with a loose pedal, that can destroy the threads. I would put some oil on the pedal threads, start it and then tighten it down with the pedal wrench (I never use a pedal wrench, I just use a standard 15 mm wrench). You should then be able to really crank it down good. If you can't crank it down very firmly then chances are you destroyed the threads in the crank and it will need to be repaired properly.

If you just run a tap though the crank arm you are just going to remove more metal from already compromised hole and it will make things even worse.