I believe that you are talking about clear anodizing, not silver.

I have three bikes with bare aluminum rims to keep polished. I've had one of them for 36 years. I agree that you should not strip the anodizing off unless you enjoy polishing rims.