Dealing with Corrupt databases


Hmm, the PostgreSQL database was corrupt and I had to manually delete a few records to get everything to the stage where it would recover it's onw integrity. So I lost a few comments. I manually retrieved some of the later ones, from google caches, and the various backups here.

So, what happened? As far as I can tell, the regularly scheduled backups at 0400 on Thursday morning caused the primary drive (old 40Gb) to fail and also completely bricked the primary backup drive (250GB). My Tertiary backup (300Gb) picked up some corruption as well, probably by copying stuff from a failing drive.

Three live copies of my data in on three physical drives and it nearly wasn't enough. However, to be fair, the stuff that was corrupted was in memory or non-comitted stuff in a database. All my client work, designs and reports are safe as far as I know.

I've whacked a new SATA drive in as primary with a fresh install of FC6, and moved services and data over as necessary. A better solution is obviously required: not just copying files, but a proper dump and restore backup, so a failed drives just means swapping connectors around, not two days of trying to remember everything I ever knew about Linux System Administration.

About Me


  • Unsolicited Bulk Email (spam), commercial solicitations, SEO related items, link exchange requests, and abuse are not welcome here and will result in complaints to your ISP.
  • Any email to the above address may be made public at the sole discretion of the recipient.

Other Stuff

  • Powered by Linux
  • (RedHat Linux)


Monthly Archives

About this Entry

This page contains a single entry by dave published on December 15, 2006 6:47 PM.

Site Outage was the previous entry in this blog.

Bra Ad of The Week ][ is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.