Cycling and bicycle discussion forums. 
   Click here to join our community Log in to access your Control Panel  


Go Back   > >

Foo Off-Topic chit chat with no general subject.

User Tag List

Reply
 
Thread Tools Search this Thread
Old 11-14-09, 03:47 PM   #1
mlts22 
Senior Member
Thread Starter
 
Join Date: Aug 2006
Bikes:
Posts: 998
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Site outage horror story

I'm glad that BF withstood the attacks, even though it took a bit to get back functioning. I'm glad it didn't suffer the horror story of another place that I used to frequent.

A small, fairly esoteric forum site which ran on a LAMP (Linux, Apache, MySQL, PHP) stack. The admins knew what they were doing, and the site was backed up to tape nightly. I didn't have admin access to the box there (nor did I want them, because with the root password comes great responsibility), but I did have moderator status on their boards, and helped with the anti-spam methods.

On a Friday night, the admins found out the machine was flattened. Apparently someone managed to get root and run dd if=/dev/zero of=/dev/whatever and zero out all the hard disks.

It gets worse. The machine wasn't just rooted at that time where everything was taken down, but weeks to months earlier. For an unknown time beforehand, someone replaced the cronjob that did the backups (via tar) with a script that did a "dd if=/dev/urandom of=/dev/tape" so the machine looked like it was backing up, but wasn't. So, the admins swapped out tapes every so often in the usual cycle of backups, thinking that the backups were doing fine.

So, pretty much all was lost from the machine, except someone who had an rsync of the bare box's config files (and thus ended up finding the changed cron job after the fact).

Moral of the story: Verify your backups on another machine, take a tape out of the rotation at least quarterly, if not monthly to put on a shelf indefinitely. Offline backups that can be set to read only (either tapes, or burning to DVD) are very important.
mlts22 is offline   Reply With Quote
Old 11-14-09, 05:02 PM   #2
chewybrian 
"Florida Man"
 
chewybrian's Avatar
 
Join Date: Mar 2008
Location: East Florida
Bikes: '66 Raleigh Superbe, 80 Nishiki Maxima, 07 Gary Fisher Utopia, 09 Surly LHT
Posts: 1,560
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 27 Post(s)
Let me see if I get your point:

The duct tape clogged the toilet and made it back up. डक्ट टेप टॉयलेट भरा है और इसे वापस कर दिया. और लीनुस और लुसी चार्ली ब्राउन के साथ फुटबॉल लात मार रहे थे. Linus and Lucy were kicking the football with Charlie Brown. शुक्रवार की रात, हम टर्मिनेटर देखा, और मशीन जीता. Friday night, we played Terminator on the XBox, and the machine won. टर्मिनेटर देखा, और मशीन जीता. सुनिश्चित क. Make sure you have backups for your offensive line, in case they get injured. के मामले में वे घायल हो गए हैं.

Did I get it right?
__________________
Campione Del Mondo Immaginario
chewybrian is offline   Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT -6. The time now is 01:12 AM.