I installed the cartridges using a socket piece that matched the outer diameter of the bearings.

And it's definitely not the rim or wheel, because I can only feel the "bump" when I hold the axle in my hand and spin the wheel. I can't feel it when the wheel is installed though. It is very minor anyways. I'll keep an eye on it and I won't worry too much unless it starts to get worse.