I think funky grime on my rim makes them look worse than they are- my braking surfaces ARE slightly worn, but I've seen far worse (as in, big grooves all the way around).
Either way, my next build/wheelset is going to be with disc brakes (this is my commuter/tourer). Hopefully discs (and avoiding potholes) increases their longevity....