I had a trek that did that. I couldn’t get the headset adjusted properly and it would pull to the side a bit when I rode it. Ended up being the fork was slightly bent to the side. Not sure how that happened, they’re usually bent forward or backwards. Maybe try putting a different fork on it to at least eliminate a bent fork.