Have you checked the alignment with the wheel installed? As you tighten the wheel you might be able to see how the wheel is pulling the frame out of alignment.