My guess is that there is some fork steerer damage, that wasn't felt initially but destroyed the headset bearings over time.