You should tighten the cap/bolt tight enough so there is no wobble when you apply the front brake and try rocking the bike back & forth holding it by the top-tube/stem. In your case this is so it's tight enough that, when lifting front wheel and angling the bike to one side, the handlebars will flop over to one side.
But no tighter than where you sense resistance in the bars tilting over in the same direction. If you still sense play in the head-set, you have a deeper problem going on. I'd recommend taking it to your LBS and asking them to diagnose your trouble.
Do get back to us. This is a mystery.