Preface: I've never messed with a headset, threaded or threadless, before and I'm figuring this out as I go along so any advice/information/links are appreciated.

Edit: So basically I've been hearing a knocking from the steerer tube area any time I hit a harsh bump and initially when I took it over to Rapid Transit a mechanic said he thought I just needed to tighten the top cap. I did that and it seemed to have gone away for a week or two. Then I rode off quite a few curbs and it started coming back but when I went to tighten it again the top cap just popped out and seems to have seized. I guess my question is whether the bike is still safe to ride if I just pound the top cap back in like I did before (no issues in the last few days other than the noise) and whether this is indicative of a more serious problem or if I just need a new top cap.

TIA,
Dan