bermudalamb/redefined-designs
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects 3 Releases Wiki Activity

docs(ops): say that the backup directories have to be created (#192)

Browse Source 
The stack gained two backup services in #147 and nothing has ever told anyone to create the directories they mount. `backup-and-restore.md` reads from both paths and the compose file mounts both, but no document creates them — while the README does exactly that for QA's data directories, ownership notes and all. Production's cutover runbook said nothing.

Hit for real during the cutover: both backup containers sat in Created, never started, and `docker logs` on them reported only that nothing matched the filter, because a container that never ran has no output. Portainer showed them beside the healthy ones and the stack looked deployed.

That silence is the reason this is worth a step of its own rather than a footnote. A backup regime that never started is indistinguishable from a working one until someone needs a restore, which is the failure mode the healthchecks in #147 exist to catch — and those healthchecks cannot fire on a container that is not running.

The cutover runbook gains the directory creation before the stack is created, and step 7 now counts containers rather than only checking the app: four, all Up, with Created called out as the thing to look for. Counted from the compose file rather than from memory — the first draft said five.

`backup-and-restore.md` gains the same note where it describes the destinations, since anyone reading that page is already thinking about paths.

No `chown`, deliberately stated: both backup images run as root, unlike the Postgres image whose data directory needs uid 999, and an unnecessary chown instruction is how people learn to run them without thinking.
This commit is contained in:
bermudalamb
2026-08-26 10:14:10 -05:00
parent 085684c1d2
commit 724e9ce19d
2 changed files with 34 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ops/production-stack-cutover.md
+27
View File
@@ -115,6 +115,28 @@ sudo docker inspect --format '{{.Config.Cmd}}' redefined-designs:latest
That `Cmd` check is worth doing. It is what makes the container migrate before serving, so deployed code can never be ahead of the schema.
## 4b. Create the backup directories
The stack gained two backup services in #147, and they bind-mount directories that no earlier deploy of this application ever needed:
```bash
sudo mkdir -p /volume1/configs/redefined-designs/backups/postgres
sudo mkdir -p /volume1/configs/redefined-designs/backups/uploads
ls -la /volume1/configs/redefined-designs/backups/
```
No `chown`. Both backup images run as root, unlike the Postgres image, whose data directory has to be owned by uid 999.
This is easy to skip because nothing fails loudly. A backup container whose mount path is missing does not crash — it sits in **Created**, having never started, and `docker logs` on it says only that nothing matched, because there is no output from a container that never ran. Portainer shows it beside the healthy ones and the stack looks deployed. The failure surfaces the day someone needs a restore.
Verify after the stack is up, as part of step 7:
```bash
sudo docker ps --filter name=backup --format '{{.Names}}\t{{.Status}}'
```
Both must be **Up**. Both will also report *unhealthy* at first and that is correct — their healthchecks assert a recent backup exists, and nothing exists until the first scheduled run. `start_period` allows 25 hours for the database and 8 days for uploads. Read those healthchecks after the first run, not before.
## 5. Remove the old stack
In Portainer: **Stacks → the stack from step 2 → Delete**.
@@ -204,8 +226,13 @@ The counts must match what you recorded in step 2. If they are zero, the contain
```bash
# Are the images still served?
sudo docker exec redefined-designs-syn ls /app/uploads | head
# Did every container in the stack actually start?
sudo docker ps -a --filter name=redefined-designs --format '{{.Names}}\t{{.Status}}'
```
Four containers, all **Up** — `redefined-designs-syn`, `redefined-designs-db-syn`, and the two backup services from step 4b. A container reading **Created** never started — that is a missing bind-mount directory or a deploy that aborted part way, and it is silent, because a container that never ran has no logs to read.
Then load the storefront in a **private window** — aggressive bundle caching on this project has produced false "still broken" reports after correct deploys. Check that product images render, sign in, and open `/admin` to confirm authentik and the admin gate still agree.
## 8. If it goes wrong