Well, this morning marks the second time that I heard a metallic rattle coming from behind and below while riding in. First time, it was the ridiculous mounting hardware supplied with my cheap rack - the nuts rattled loose within 30 minutes. I replaced them with longer bolts and double-nuts with lock-washers in between.
Today I heard another strange rattle (again about 30 minutes in) and stopped to check it out - the bolt that holds the top-rear hook onto my right (chain-side) pannier had vibrated itself loose! Had I not stopped when I did, the bit of nylon webbing that it was dangling from would have dropped the hook and bolt somewhere along the MUP and my pannier would be swinging around off of just one. This would open the possibility for the bag to swing into my spokes, or derailleur!
The moral of the story is - if you hear something that sounds loose on your bike that wasn't when you started out - always take a moment to stop and try to figure it out before you loose parts and get hurt.