diff --git a/docs/ops/production-stack-cutover.md b/docs/ops/production-stack-cutover.md index ee0b0cc..75b0339 100644 --- a/docs/ops/production-stack-cutover.md +++ b/docs/ops/production-stack-cutover.md @@ -135,7 +135,33 @@ Verify after the stack is up, as part of step 7: sudo docker ps --filter name=backup --format '{{.Names}}\t{{.Status}}' ``` -Both must be **Up**. Both will also report *unhealthy* at first and that is correct — their healthchecks assert a recent backup exists, and nothing exists until the first scheduled run. `start_period` allows 25 hours for the database and 8 days for uploads. Read those healthchecks after the first run, not before. +Both must be **Up**. Both will also report *starting* rather than healthy, and that is correct — their healthchecks assert a recent backup exists, and nothing exists until the first scheduled run. `start_period` allows 25 hours for the database and 8 days for uploads. + +**Do not wait for those runs to find out whether this works.** Up and scheduled is not the same as producing a restorable file, and the difference is one command each: + +```bash +sudo docker exec redefined-designs-db-backup-syn /backup.sh +ls -lht /volume1/configs/redefined-designs/backups/postgres/daily | head + +sudo docker exec redefined-designs-uploads-backup-syn backup +ls -lht /volume1/configs/redefined-designs/backups/uploads | head +``` + +Check the sizes are plausible. A dump of an empty database succeeds and tells you nothing, so confirm the dump carries rows rather than only a schema: + +```bash +zcat /volume1/configs/redefined-designs/backups/postgres/daily/redefined-latest.sql.gz \ + | grep -E "^COPY public\.(items|customers|orders)" +``` + +Then confirm the healthchecks agree with where the files landed — a check whose `find` path disagrees with where the tool actually writes reports unhealthy forever, and is worth catching now rather than in a year: + +```bash +sudo docker inspect --format '{{.Name}} {{.State.Health.Status}}' \ + redefined-designs-db-backup-syn redefined-designs-uploads-backup-syn +``` + +Both reach `healthy` within their check intervals once a file exists — an hour for the database, six for uploads. ## 5. Remove the old stack