Yes, the entire piece (alloy) will have to be addressed, if you wish to polish out those spots of oxidation.

I use Extra Strength "Eazy Off" and do so with great care. Disassembly the calipers. Brush some EZoff on a piece and give it a little while to work (we are talking a minute or less - left on too long, the caustic in the EZoff will attack and eat the aluminum. With that in mind, use the EZoff, wait and then scrub with a plastic brush (tooth brush is one choice, but I use a really soft brass brush). Rinse with cool water thoroughly and look at the result. It will take a few passes to get all of the anodizing off.

Once the anodized coating is gone, use very fine wet and dry sand paper to polish the piece to a mirror finish. Start with #600 and work up to #1500 or finer. You will see the result materialize, as you work.

Good luck, should you wish to try.