I took the advice of jeff wills and found a tool to tighten it down properly. I also put a good amount of teflon tape on to the threads of the bolt. It held perfectly tight for over a week now so things are looking good. If it comes loose again, I won't be using jb weld since the consensus seems to be against it.
Thanks for your help.