Most of that glue looks like it would just flake off. A wire brush (brass) is helpful. There's no sense in getting carried away with removal, you don't need a claen surface to glue.
To remove anodizing, use EZ Off oven cleaner, not the new 'eco friendly' stuff, but the old fashioned stuff that has lye, or any off brand that contains mostly lye. Be careful, especially with your eyes. I end up with a finer grit paper, more like 1000 wet, and then to the buffing wheel after that.
Good advice from a master here:
http://www.raydobbins.com/polishing/