Thanks for linking in the pictures. That'll help everyone see what we're talking about here.
There's no point rebuilding as the steerer tube is bent too.
About 2 inches up from the base of the steerer the tube changes wall section - where the ID (internal diameter) changes from 18.9 to 22.2 - at this change in wall section there's a bend. With a quick bit of mathsthe bend looks to be approx 1 degree, or 1mm in the 2 inches to the bend point
Next check is to see if that has translated to the frame too, that's this evenings job.