It’s not strictly a matter of the material so much as how the material is used. For example, a thin walled frame will see higher stress than a frame with thick tubes. From a corrosion standpoint, the coating process makes a difference. The joining process is critical as well; properly executed welds are critical and I suspect, more durable than bonded joints.
In a general sense I’d say that welded Ti is going to near the top of the durability scale since it has a high fatigue limit and does not corrode in any meaningful way. Steel also has a high fatigue limit but corrosion can be an issue depending on how the frame is treated. Aluminum has no fatigue limit thus it is prone to cracking. Carbon is prone to impact damage, UV aging, bonded joint failure, and other things I’m sure. Not saying carbon is bad, but I couldn’t put it in the same durability class as Ti or steel.