Barring spokes pulling through the rim, a rim wears out when the brake tracks get so thin that the tire pooches the rim bead out and the tire will eventually pop off disastrously, usually when braking. This happens much much sooner if you ride in the rain a lot, particularly in hilly terrain. You'll usually have a warning: a pulsing of your brake lever during gentle braking. Upon inspection, you'll see that the brake track is very concave and that the bead above the brake track is leaning out at an alarming angle, usually just in one spot. You can usually get home as long as you go slow and don't use that brake again. Time to replace the rim.