Is there any difference in expected life of a frame based on what material it's made out of?

Is there a point where a frame is worn out and no longer safe to ride? Is this based on miles?