ADR-0060: Backup & Restore — Scripted Ordered Backup with Automated Restore Rehearsal¶
- Status: accepted (realization deferred — the arc is scheduled by Nicolas, not started)
Status note (2026-09-04, false-by-time): "not started" stopped being true long ago and was never corrected.deploy/backup.shanddeploy/verify-backup.shboth exist, run on the reference server, and have produced dated verification records — including one recorded FAIL (feedback-2026-08-22 #12), which is the control working. The 2026-08-20 hardening wave added full archive validation before the destructive point and made verify-backup grade against the manifest (old-PASS → new-FAIL proven). Still genuinely open, and tracked in v1.0-criteria.md C-5: the §9.3 second-machine restore rehearsal with a dated PASS record, and §9.1's two host-side gaps (off-box target, dead-man ping). - Date: 2026-07-12
- Deciders: Nicolas Burri ("I have read your decision proposals and agree with all of them"), on the worked concept in backup-restore-design.md.
- Relates to: ADR-0001 (content-addressed blobs — the ordering rule's foundation), ADR-0018 (RPO ≤ 24h / RTO ≤ 1 business day / off-box copy), ADR-0025 + REQ-DPR-011 (bounded backup retention), ADR-0040 (legal holds do not extend backups); REQ-BAK-001..005, REQ-NFR-004.
- Refined by: ADR-0116 (the primary mechanism moves
in-app — the app executes its own audited backups;
deploy/backup.shbecomes the drilled belt-and-braces fallback, and the ISO 13485 records-control evidence becomes native audit events rather than a shell log) - Regulatory anchor: ISO 13485 §4.2.4 g) (prevent loss of documents) and §4.2.5 (records stay retrievable) — for a QMS living in LQMS, the backup system IS that control, so restore must be proven, not assumed.
Amendment (2026-09-04 — recording ADR-0116). §1's mechanism decision is demoted, not repealed. Since 2026-08-15 the primary backup mechanism is the app's own audited act (
BACKUP_TRIGGERED/BACKUP_RECORDED, V102, theNO_VERIFIED_BACKUPfinding), which moves the §4.2.4 evidence out of the shell log and into the trail. Everything else here survives intact and is what ADR-0116 leans on: the blobs-first / database-second ordering rule (§1), the restore-verification discipline and its record chain (§2), the bounded-retention and re-disposal interplay (§3), the encryption posture (§4) and the deferred register (§5). Restore stays deploy-level forever — ADR-0116 §3 makes that the load-bearing split, and this ADR's runbook is where it lives. Two accepted ADRs owned the same ISO 13485 control with no link in either direction until the 2026-09-04 coherence review named it.
Decision¶
- Mechanism (B-1a):
deploy/backup.sh— nightly cron; blobs first, database second (the ADR-0001 ordering rule makes the archive self-consistent without downtime); blob re-hash in the copy pass (REQ-BAK-004, doubling as primary bit-rot detection);pg_dump -Fc; Keycloak realm export (users included); manifest (timestamp, app git SHA, blob count/bytes/ hash sample, dump + export checksums, retention window); one self-contained zip (REQ-BAK-001 wording stands unamended — D-1); off-box copy to a configured target (ADR-0018); prune to the bounded window, default 35 days, configurable and documented (D-2); dead-man success ping (absence of the ping is the alert). - Restore verification (B-2a — the differentiator): weekly
deploy/verify-backup.shrestores the newest archive into a scratch compose project and verifies: migrations current, app healthy + manifest SHA match, blob re-hash clean, referential closure (every DB blob reference resolves), search functional; writes a dated verification record. The record chain is the audit answer to "when did you last prove a restore?" and is cited evidence in the CSV package (ADR-0061). - Disposal interplay (B-3): bounded retention + deployment.md statement (disposed content
persists in backups at most
<window>days; legal holds don't extend backups); the restore runbook carries a mandatory re-disposal step (replay disposals recorded after the archive's timestamp — the audit trail survives disposal and drives the replay). - Encryption: platform-level at rest now (ADR-0018 posture); explicit archive encryption (age/gpg) is the registered option when the off-box target is third-party storage (D-3).
- Deferred, registered: per-scope export bundle (B-4, REQ-BAK-002/005) rides the
inspection-copy/report wave; pgBackRest/WAL PITR when a VM deployment becomes real; Keycloak
backed by PostgreSQL is the production-shaped answer that dissolves the realm-export special
case (D-4, ties into hosting). The
lqms-contentvolume is in scope of every full backup (Nicolas 2026-07-11: images live there — "we will have to consider this for the backup").
Consequences¶
- Sequencing (D-5): B-1a + B-3 docs first (deploy-side, no product code), B-2a right behind; nothing blocks current work. Realization details live in the planning doc — this ADR records the decision so the arc survives until scheduled.
- REQ-BAK-001/003/004 + REQ-NFR-004 + REQ-DPR-011 become covered when the arc lands; BAK-002/005 stay honest gaps until B-4.