Thank you all for the suggestions.

I'm inclined to first file down the caliper housing so I can fit it without the spacer. Just knowing it is there bothers me anyway

If problem persists, I will swap out pads next.