A backup that has actually been restored, and pg_wrapper's argv[0]

A backup that has actually been restored, and pg_wrapper's argv[0] Go-live item 2. `dr_state` has existed since the first enterprise migration, with a comment on `last_verified_restore_at` saying it "is set only by an actual restore drill, never by a successful backup", and §17 explaining why: "the gap between 'we take backups' and 'we have restored one' is where DR plans fail." Nothing wrote a single column of it. Nothing took a backup. The table was an argument. ## What `dam-backup` owns, and what it refuses to The per-tenant half, which is the whole D2 argument: `pg_dump --format=custom` of one schema, uploaded to `backups//-.dump` — outside every tenant prefix, because a backup filed under the tenant it protects is one a lifecycle policy can tier into Glacier, and a backup you wait twelve hours to read is not a backup you can restore from during an incident. It does not own WAL archiving, PITR, S3 versioning or cross-region replication. Those are infrastructure, §17's five-minute RPO comes from them rather than from here, and an application reimplementing them would be a worse version of both. Said plainly in ARCHITECTURE §17 and `docker/DEPLOY.md` rather than left for somebody to assume. ## The drill is the point `restore-drill` replays the latest dump into a scratch schema, counts the assets against the number recorded in the object key, and drops the scratch schema — then writes `last_verified_restore_at`. It is the only thing that writes that column. The asset count rides in the key rather than being looked up, because a backup that can only be validated by consulting the database it is a backup *of* is not much of a backup. The live schema is renamed aside and back rather than restored over. A drill that can damage the thing it verifies is one nobody runs on production, and a DR mechanism nobody exercises is the state this commit exists to leave behind. The swap-back is deliberately not written as `?` on the preceding line, so it happens on the error path too. Two cases are the ones worth having: a backup must **not** move the verification column, and a drill whose restore returns the wrong count must **fail**. `pg_restore` exiting zero proves the file parsed, not that the data arrived. ## Running it in the image found what the laptop could not Debian's `/usr/bin/pg_dump` is a symlink to `pg_wrapper`, a multi-call program that decides which versioned binary to exec **from `argv[0]`**. The toolchain canonicalised its paths — copied from `dam_media::vips`, where it defends against a symlink swapped between discovery and use — which made `argv[0]` `pg_wrapper` and left it unable to tell what was asked of it. Every backup taken inside the container failed with `io: os error 2`, while the conda-installed binaries on the laptop, which are not wrappers, worked perfectly. The canonicalisation is gone for these two tools with the reason written down. The defence it gave up is small — the paths come from `PATH` or `DAMRS_PG_BIN`, both operator-controlled — and the loss was total. ## Verified, twice Against the real 185-asset library from the laptop: backed up, restored, counted, live schema intact at 187 rows and 488 derivatives, no leftover schemas, `dr-report` green. Then the same two commands from inside the image against the same database. `pg_dump` is pinned at 17.11 in `.mise.toml` — matching the image's 17.11, and a major at least the server's, since pg_dump refuses a newer server. The Dockerfile prints its version at build time next to the vips and ffmpeg ones, so a base bump that moves the client behind the server shows up in a build log rather than in a failed restore. `dr-report` exits non-zero while any tenant is unverified, so it is usable as a check. A DR report that cannot fail a pipeline is one that gets skimmed.