I got impatient and tore it all apart. It's definitely the needle bearing which has the roughness. The two cartridge bearings are perfectly smooth. I guess I'll try them out on the road and see if it "polishes out" and if not I'll swap in some new HK1010's :/