Sounds like a standard headtube that's a bit worn or a bad cup. Have you tried a few different cups to make sure that's not the problem?
If the cup pushes in by hand but is snug (loose but no play), there's no reason not to install it with some red Loctite. First clean all the surfaces really well with acetone or brake-clean. Install the cup with Loctite and clamp in place with a headset press or C-clamp and some wood cauls. Let it set-up at least a few hours or overnight before assembling and riding.
I've done this on a few bikes and the cups have never moved in use, and they come out without issue when you want to remove them.
Again, this is only a good fix for a cup that fits loose but without any play.