If the brakes are tearing into the side of the tire, then they are adjusted too high.

Move them down slightly on their mounts.

The rim should be fine, unless it starts looking a bit concave from lots of wear. If the pads pick up little bits of metal or rock they'll tear up the side of the rims, but usually it is just superficial.

Replacing the pads (and making sure they're adjusted properly) is a good idea.