Turns out the derailleur itself was bent. Not sure how I did that, I do have a replaceable hanger and that wasn't bent! Thank you for all the suggestions.