Just make sure the material that holds your cleat to the pedal isn't too thin. The torn up part doesn't matter that much unless it dramatically thins the material that your cleat needs to hold the pedal.
I cracked one cleat when I fell. I cracked one when I over-tightened it. I shaved enough off one pair in a crash enough so that I replaced the cleats (the material got worn away). Even with that I'm on my third pair of cleats (Keo, grey) in 4 years.
I don't walk around much on my cleats. I tend to track stand.
cdr