It is better when the wheel is out and spin it in my hands but it's still not good.

Took it apart again today. Seems it is very sensitive to the two nuts on the torque arm side of the hub. These need to be very loose, but there is a fine line between having them loose enough to allow the hub to spin nicely, and tight enough to limit play in the hub and bearings.

I've also noticed that I'm missing a spacer where the hub torque arm attaches to the frame. Not sure where I'll find one of those or even what size it should be.