I seem to remember some handlebars not tightening fully being an issue on a bike before, but can't remember which brand.
On my recumbent I don't really push back on the seat as much as I spin the pedals. No pulling on the handlebars either, but they shouldn't move in the stem if you do. If the bar is too small for the stem to properly clamp it, then maybe a metal shim in there might work.