CXwrench is right, a primary purpose of thru axle is to allow the consistent realignment of the disc rotor in the caliper, the hub should barely be bigger then the axle so there isn't room to wiggle the wheel when tightening. When installing the wheel I either have the bike upside down and once the thru axle starts to thread in the other side I let go of the wheel or I have it in the stand and same thing. The result is always right where it belongs. Although it shouldn't matter, since it does, make sure you're not holding the wheel causing it to shift. It may be if a factory hub its not a perfect tolerance or the thru axle isn't quite right. If the current solution is to just not hold the wheel then go with it.