Backup & restore
make backup dumps each engine data-only; make restore-drill restores the newest dump into a scratch namespace and runs the full prover against it.
A backup you have never restored is a guess. The drill restores every dump into a throwaway database, re-checks every money invariant against it, and deletes the throwaway — so you know the backup works before you need it.
Source scripts/backup.tsscripts/restore-drill.tsscripts/support/db-tools.ts
A backup that isn’t provably restorable is a hope, not a backup. The lab ships both halves:
make backup dumps, make restore-drill proves the dump restores — into a scratch namespace,
under the full prover, with the scratch dropped afterward.
Backup
make backup writes a data-only dump per env-named engine into backups/:
- postgres:
pg_dump --data-only --no-owner --schema=public --disable-triggers→backups/pg-<stamp>.sql - mysql:
mysqldump --no-create-info --single-transaction --replace --skip-triggerswith the table list parsed fromdb/mysql-schema.sql→backups/mysql-<stamp>.sql, so a shared database’s unrelated tables stay out.
Data-only is deliberate: DDL comes from the repo’s canonical db/*.sql at restore time, so
the drill also proves the dump still matches the current schema (schema_meta rides in the
data). The trigger bypass is safe because nothing is trusted to it — the prover re-checks
every invariant the triggers enforce. An engine the env names that fails to dump fails the
run; a backup that silently skips is a hope again.
The drill
make restore-drill takes the newest dump per engine and closes the loop:
- Create a scratch
el_drill_*database. - Apply
db/*.sql, then clear the seeded platform rows (postgres; the mysql dump usesREPLACE) so the dump’s values land, not the seeds. - Apply the dump, open a real store on the scratch, and run the full prover:
conserved,backed,noOverdraft,chainIntact,consistent— PASS requires every one true. - Drop the scratch in a
finally. A killed drill leaves anel_drill_*name thatmake db-cleansweeps.
Run the drill after every backup-tooling or schema change and on a schedule; a drill that has not run since the schema last changed proves nothing.
A real restore
The drill is the rehearsal; a disaster restore is the same moves against a fresh production database, minus the drop:
- Stop the worker and the API — no writes during restore.
- Create the database, apply
db/*.sql, clear the seeded rows. - Apply the chosen dump and point
DATABASE_URLat the new database. make provebefore taking traffic; then restart and watch the firstworker.checkpoint.verifybeat.
The backup contains the sealed checkpoints, and the prover re-derives the chain against them — so a restore is only as trustworthy as the dump’s custody, which is what shipping dumps and the audit trail to write-once storage is for. Losing the tail between the last dump and the incident is expected; the chain makes the cut point provable, not invisible.