After I overhauled the bearings to my threaded headset it keeps coming loose. I am following directions from Park tools blue book, where you incrementally tighten the threaded race, tighten the locknut, test for knocking, repeat until knocking is eliminated. Each time after I finish the headset is good and tight, testing by rocking the fork as well as holding the front brake and rocking downwards on the handlebars. Then I take it for a test ride, and as soon as I hit a bump there is a weird hollow sound from the headset. Then when I test it again it is loose! I have adjusted it 3 times and this keeps happening. Well now I have some questions which I was wondering if anyone could help with:

#1: There are basically 4 different possible configurations for the two bearing cages... is it possible that the headset will only stay tight with both in their previous positions? Unfortunately I did not make a note of how it was set up before. Park Tools says "Retainers have only one correct orientation. The open side of the ball retainer should face the cone shaped race, not the cup shaped race." This is how I have them right now and it isn't working...

#2: While I was cleaning the cages some of the balls popped out. This was my first attempt at overhauling a headset and I was surprised that the balls came out, since I am more used to the bottom bracket cages where the balls pretty much stay in. I put the balls back in and used them anyhow. Is it possible that these retainers are the problem?

#3: Does anyone have any other suggestions or advice? It would be greatly appreciated

Before I got the bright idea to overhaul the headset it was staying tight without a problem, so I don't think it's the headset itself that is the problem. I think the problem is probably my own limited mechanical ability and general overall incompetence I sure would appreciate your suggestions.