You're referring to flex when you close the quick release?

That suggests to me that the axle spacing is too narrow. Put a washer under the locknut on each side to make the axle spacing match the fork end spacing.