as a short bicycle rider: you're probably running into a deformation above the brake hole, bur or otherwise. if not that, i have had a few frames where the tube tapers. however, i don't know if steerer tubes were ever butted (even if they were, you would be going from thick-thin-thick, so it should fit. try fitting the quill stem upside down, to check for roundness?