The last couple of rides I have been on, I have noted my headset getting loose. I would get back, tighten it where the wheel would have some resistance, back it off a touch and tighten everything down. Today I go out, got about five miles from home and noted my headset knocking really hard. As I was going up a hill my whole stem pulled up off the steerer about 1/4". Wheel didn't turn, lucky for me, and I was able to limp back home. I pull everything apart to give it a look and all seems fine aside from the fact that the stem lock (w/e it's called) has pulled out of the fork steerer.

I check everything over again, put it back in and tighten it down to spec. Every time I tighten the stem cap, it loosens the lock. I have even tightened the lock a bit past spec and am still having the issue, I cannot get the headset tight enough to stop the front end from knocking, or even to the point of making the front wheel harder to turn. Is there something I am missing here?