Eight Steps to Bullet-Proof Database Disaster Recovery
October 20–23
Disks fail. RAM goes bad. Software bugs introduce silent corruption that spreads through a cluster before anyone notices. The question is not whether this will happen. The question is whether you have a procedure ready when it does. I have a procedure. Eight steps, in the order you actually need them during an incident. Step one: detect. Log patterns, checksum mismatches, page header anomalies, WAL inconsistencies. Step two: stop writing immediately. Every second the cluster stays up with damaged files makes recovery harder. Steps three and four are backup restoration and point-in-time recovery. I spend time on the part most people skip: how to verify that your backup is actually intact before you bet your data on it. LSN markers and timestamps give you precise control over how far to roll forward. Step five is salvage. Not every situation has a clean backup. Sometimes you need to extract individual tables with pg_dump or inspect pages at a low level to pull out what you can. Step six is pg_resetwal, which I only use as a last resort. I explain what it actually does to the WAL state and what you are agreeing to lose. Steps seven and eight are about never being here again. Checksum enforcement, scheduled integrity checks, WAL archiving discipline, and replica placement for failover. Every command in this talk runs on standard PostgreSQL. No third-party tools required.
