Thanks for your help.
I took it apart and reassembled it. The cartridge looked like it is sitting pretty square, I tighten everything up nicely and the gap is smaller now. I can squeeze my nail in the back but in the front my nail wiggles a little bit. So perhaps an extra mm in the front.
On the 2nd pic, they are not gaps between the spacers, but they didn't line up perfectly.

So I guess it's fine now. I must have messed it up when I upgraded the stem and bar. Thanks again for your help.