main
17
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 synAdminandClaude Opus 5
|
9cc82002b8 |
feat(auth): groundwork for signing in with Google (#340)
Nothing a customer can see. The schema change and the configuration land on their own so the widest-reaching edit in the project can be reviewed for what it is rather than buried inside a feature. The password hash becomes nullable. That is one line and it is not the work; the work is that every read of the column is now a question rather than a fact. Three places compared against it with bcrypt, and all three now ask first through one shared function. That function exists because the alternative is worse than a wrong answer. bcrypt.compare throws on a null hash rather than returning false, so any call site that forgot the check would answer a sign-in attempt with a 500 instead of a refusal. On the login route that is also an oracle, because it would happen for exactly the accounts that have no password. One function rather than a null check repeated three times means the question is asked identically everywhere and a fourth site cannot forget to ask it. Nothing writes a null yet. The first accounts without a password arrive with the sign-up path, which is why this is landed ahead of them. The identities table is a table rather than columns on customers, because one customer may eventually hold more than one. Columns would make a second provider a migration and a third an embarrassment. Its important column is the provider subject, and the comment on it is the whole security posture of the feature in one place: never the email. An email is a display value its owner can change and a provider may reassign; a subject is opaque and stable for the life of the account. Matching on the email would strand a customer who changed theirs and, far worse, hand their account to whoever inherited the old address. Unique across the provider and subject together, not the subject alone. The down migration drops the table and deliberately does not restore the NOT NULL. Re-adding it fails outright once a passwordless customer exists, and a down migration that destroys accounts to satisfy a constraint is far worse than a column that is merely more permissive than it needs to be. The redirect URI is derived from PUBLIC_URL, the same single source the WebAuthn Relying Party ID uses and for the same reason: Google compares it as an exact string and answers a mismatch with a message that says nothing about which half is wrong. Deriving it means the value is correct by construction anywhere the email links already are. The tests are mostly about what must not end up in it, since a trailing slash on PUBLIC_URL is an easy way to produce a URI that is one character from the registered one. The config also reports whether it is enabled at all, so a developer without credentials gets a storefront that works and simply does not offer the button, rather than one that offers it and fails. Absent rather than disabled, the same choice made for a browser without WebAuthn. Environment validation refuses to boot on one credential without the other, matching how the SMTP pair is handled. Half-configured is the case worth catching because the failure otherwise arrives at the moment a customer presses the button. The QA compose file sets both to empty, and the comment there says why at length rather than leaving it to look like an oversight. Google refuses a redirect URI whose host is not under a domain whose ownership has been proved by DNS, and nobody can prove ownership of anything under bermudalamb.synology.me because Synology owns the registrable domain above it. That is the same wall #285 hit with Cloudflare. So QA cannot run this at all until #313 moves it to a subdomain of the real domain, at which point it is two stack variables and one console entry, with no code change either way. Also corrects the record in #332, which lists account deletion as confirming with a password. It does not; the route takes none and the confirmation is a modal in the account page. Deletion needed no change here. Verified: backend tsc clean for src and tests, 550 unit tests pass including new coverage of the config derivation, the null-hash comparison and the environment rules; lint clean apart from warnings that predate this branch. The integration suite needs a database this machine has no Docker for. Closes #340 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
synAdminandClaude Opus 5
|
ac3f6e91f5 |
feat(analytics): report consenting customers' activity to Brevo (#56)
Loads Brevo's web tracker for a signed-in customer who has consented, reports route changes as page views, and tracks the three events the issue asked for: added_to_cart, favorited, and checkout_completed. The four design questions were settled on the issue in August and this implements those answers. The consent gate is the part worth reading. The decision recorded on the issue was "gate it behind consent", but the sentence customers actually agreed to named only email: "I want to receive occasional emails about new one-of-a-kind items". Gating a tracker on `marketing_consent` while that was the stored wording would have treated "email me about new items" as authorisation to send someone's browsing to a third party, which it does not say — and this project stores the wording verbatim against each customer precisely so that a record says what the customer saw. So the sentence is widened here, and the tracker is gated on `analytics_consent`, a field the server computes by comparing the wording stored against a customer with the current constant. Changing the sentence therefore does not retroactively widen anybody's consent: everyone who agreed to the old text keeps their email consent and is not tracked until they re-consent through the account page. A boolean alone could not tell those two populations apart, which is the whole reason the text is stored per customer. `analyticsConsent` is exported and has its own unit test, because "agreeing to the old wording does not authorise tracking" is the rule that silently tracks people if it regresses — their flag really is true. QA stays out of the live Brevo account by construction rather than by remembering. The key is per-environment, the tracker never loads without one, and `docker-compose.qa.yml` sets an empty literal with no stack variable behind it, so nothing can inherit a value from the host or be pasted in from production's stack. Same reasoning as QA_DB_PASSWORD and the QA_SMTP_ names beside it. Events are reported from the API layer rather than the UI call sites, so no caller can add to the cart or favorite an item without it being counted, and each fires only after the response was accepted — a refused add is not reported as one. The two checkout completions each name their processor, because a demo purchase charges nothing and counting it as a sale would overstate revenue. The privacy policy gains an analytics section in this change rather than a follow-up, since the published policy previously described none of this and would otherwise have lagged the code. It is deliberate about the limits: withdrawing consent stops further reporting, but anything already sent stays with Brevo, and a script already injected cannot be un-injected — `stopBrevoTracking` stops calls, it does not unload sa.js. That is said in the code too, because "tracking stops" reads as a stronger promise than any web tracker can make. Verified: backend tsc clean, both lint suites 0 errors with no new warnings, 474 unit tests passing across 33 suites, and the frontend production build green including the compose-environment guard. Not verified: integration and e2e, which need a database and a Node this machine does not have active, and no real Brevo key was exercised — the tracker has never been observed reporting to an actual account. Closes #56 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
 bermudalambandClaude Opus 5
|
66cafeb89c |
docs(compose): keep the prod comment block's sentences with their own variable (#281)
ANTHROPIC_API_KEY's explanatory paragraph already had a dangling continuation trailing after later entries. When REMBG_URL was added, its entry was inserted ahead of that continuation, so the file read as though "put a spend limit on the key in the Anthropic console" belonged to the background-removal sidecar rather than to Anthropic. This file is read during the cutover runbook, so a misattributed sentence there is not just cosmetic. Reordered the comment lines so ANTHROPIC_API_KEY's full paragraph is contiguous and REMBG_URL's own two-line entry stands on its own at the end. No environment: line was touched — only the comment block above the services: section. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
bermudalambandClaude Opus 5
|
61e9c239d3 |
feat(intake): talk to the rembg sidecar, always naming u2net (#281)
Adds the client that will let the intake path remove backgrounds from submitted photos via the rembg sidecar over HTTP. isRembgConfigured() reports whether REMBG_URL is set (unconfigured is a normal, working state, not a failure), and removeBackground() posts a file to /api/remove and resolves with the PNG bytes it gets back, rejecting on every failure — unconfigured, unreachable, a non-2xx response, or a body that fails the same magic-byte PNG check the upload path already uses. The one hard rule: every request names model=u2net explicitly and this is never configurable. The sidecar's default model, reached simply by omitting the parameter, is bria-rmbg, which is licensed non-commercial — a licensing problem that a shop cannot silently ship, and one that would produce a perfectly good image with nothing in it to reveal the mistake. The test that posts against a real stub HTTP server and asserts model=u2net appears on the wire is the only thing guarding against that regressing. Wires REMBG_URL into both docker-compose.qa.yml and docker-compose.prod.yml as an optional variable, right after ANTHROPIC_WORKSPACE_ID, following the existing style in each file's environment block and header comment. It is deliberately left out of envValidation.ts's ALWAYS_REQUIRED — requiring it would make an environment with no sidecar refuse to boot, which is exactly the failure mode this feature is designed to avoid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
bermudalambandClaude Opus 5
|
6c6aaa46eb |
fix(intake): send the workspace id an identity-linked key requires (#271)
Every draft in QA failed with a 400: "anthropic-workspace-id is required when authenticating with an identity-linked API key". A key issued against a workspace, rather than standing alone, is refused unless the request names the workspace it acts in — and the client was constructed with an API key and nothing else. Nothing about a key's shape says which kind it is, so no amount of configuration checking would have caught this. Only a real call would, which is exactly what #223's task 8 existed to make. Sent only when ANTHROPIC_WORKSPACE_ID is set. Plenty of keys need no workspace, and sending an empty header would turn the ordinary case into a different error rather than leaving it working. Both compose files carry it with an empty default so an unset variable cannot fail a deploy, and the cutover doc goes from fifteen interpolated names to sixteen — checked against the file, and every name in the list now matches one in the compose. The failure handling needed no change and got none. The submission kept its photos, the draft recorded ai_error, and the review queue showed the reason. A model call failing must never lose somebody's consignment, and it did not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
bermudalambandClaude Opus 5
|
7b4fbbb9e3 |
build(intake): give the containers an INTAKE_ACTION_SECRET (#224)
A gap in the plan rather than in the code: nothing wired the secret into either stack, so the feature would have shipped with its signed links permanently disabled and nothing saying why. Both compose files now interpolate it, with `:-` so an unset variable stays empty rather than failing the deploy. QA takes QA_INTAKE_ACTION_SECRET, its own value rather than production's, for the same reason as every other QA_ prefixed credential — and more sharply here, because a link signed with it acts on a draft without a login. Rotating the secret revokes every outstanding link, which is the intended answer to one leaking. The cutover doc counted fourteen interpolated names and now counts fifteen. That document says it is checked against the file rather than from memory, so it was: fifteen in the compose file, the same fifteen listed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
bermudalamb
|
2424305d43 |
build(intake): give the container an ANTHROPIC_API_KEY (#223)
The drafting worker needs a credential, and a Portainer stack variable alone does not reach the container — stack variables are interpolated into the compose file as ${VAR}, and a service receives exactly what its own environment block lists. That is how UPLOADS_DIR went missing in #118, and both files say so; this adds the line that makes the variable actually arrive.
QA takes it from QA_ANTHROPIC_API_KEY, prefixed like the database and SMTP credentials so production's key cannot be pasted there and silently work. It is also worth a key of its own rather than sharing production's, because this is the only credential in either stack that spends money per call, on a path anybody holding an upload link can trigger.
Absent is a working configuration in both, deliberately, which is why prod's line carries `:-` and neither variable joins the always-required list. A submission still arrives, keeps its photos and waits undrafted. Losing somebody's consignment to an expired key would be far worse than an item arriving without its description written, and the photos may be the only copy of something no longer in the sender's hands. USPS is the existing precedent for a credential whose absence degrades rather than fails.
The comments say plainly that a spend limit belongs on the key in the Anthropic console, since nothing in this repository can enforce one and #227's submission ceiling bounds the volume rather than the bill.
docs/ops/production-stack-cutover.md said the compose file interpolates thirteen names and listed them. It now says fourteen, because that document stakes its usefulness on being checked against the file rather than written from memory — a cutover working from a stale list is how a variable gets left behind, which is the failure the document exists to prevent. Counted from the file: exactly fourteen.
composeEnvironment.test.ts passes, 24 tests. It checks that every deployment sets what the validator requires, so adding a variable ahead of a validator entry cannot break it — the entry itself comes with the worker.
Ref #223
|
||
|
bermudalambandClaude Opus 5
|
70d04186b1 |
docs(ops): stop the compose header contradicting itself, and name the four variables step 2 dropped (#204)
Three corrections a review of #196 found, all in text #196 itself rewrote. **The compose header asserted something it falsified three lines later.** "All must be set in Portainer for this stack. All are secrets except DEMO_MODE" — but three PayPal entries are unused while `DEMO_MODE` is `true`, three more are marked Optional, and `SMTP_FROM` is not a secret. The runbook sends the operator to that exact block as authoritative, so an operator cutting over during the demo interim reads "all must be set", has no live PayPal credentials — which is the whole reason the interim exists — and either stops or invents a `BACKUP_PASSPHRASE`, which is how you get archives nobody can decrypt. The blanket claim is gone; each entry already says whether it is required and when. **Step 2 enumerated nine of the thirteen interpolated names.** `USPS_CLIENT_ID`, `USPS_CLIENT_SECRET`, `UPLOADS_BASE_URL` and `BACKUP_PASSPHRASE` were missing. The USPS pair is the one that matters, and it now gets a sentence of its own: losing it is the only silent failure in this step. Address validation is skipped when those are empty rather than failing, so checkout keeps working and quietly stops validating addresses, with no crash loop and nothing in step 7 that would notice. The list also now says to take everything the stack holds rather than working from the list, because an enumeration reads as a checklist however it is introduced. **The `DB_PASSWORD` failure was described wrongly, and its error names a variable the operator never typed.** It does not fail to authenticate against its data directory — the app never reaches a connection attempt, `checkAlwaysRequired` refuses at boot, and the message says `PGPASSWORD` because the compose file injects it as `PGPASSWORD=${DB_PASSWORD}`. An operator grepping for `DB_PASSWORD` finds nothing. That is now stated. The crash-loop examples were reordered to match the case the section claims to be about. It headlines "the likeliest outcome of a missed step 2", but the only block shown was the PayPal triple, which cannot occur during the demo interim, and #196 replaced it with a `DEMO_MODE`-only block that is not what a missed step 2 produces either. A wholesale miss is two problems led by `PGPASSWORD`; that is now first, with the `DEMO_MODE`-only and mistyped-value forms after it. Every quoted line was verified by running the real validator against production's entry set rather than by reading the source — `2 problem(s)` and `1 problem(s)` counts included — and all four now match character for character. The mistyped-value message is quoted in full rather than truncated, which was the point of the complaint that produced it. `DEMO_MODE` is no longer called "the only one that is not a secret", which was false of `SMTP_FROM` and `UPLOADS_BASE_URL`. It is the only one that is a setting rather than a credential, which is true and a better hook. Verified: 278 backend unit tests pass, including the compose guard that parses this file. Closes #204 Refs #196 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
bermudalambandClaude Opus 5
|
2fe3aa055f |
docs(ops): make the cutover runbook agree with the compose file about DEMO_MODE (#196)
#190 turned `DEMO_MODE` from a hardcoded compose value into a Portainer stack variable with no default, and updated the compose header. The runbook that actually creates the stack was not updated with it, so the document and the file it deploys have been disagreeing since — in the three places most likely to be read under pressure. Step 2's list of stack variables to record did not include `DEMO_MODE`, and step 6 says to add the variables from steps 2 and 3. An operator following this literally creates a stack that cannot boot. It is now in the list, with a paragraph of its own: it is the only one of the nine that is not a secret, which is exactly why it is the easy one to skip past. The troubleshooting section said `DEMO_MODE` was hardcoded and therefore could not be missing, so its absence from the error list proved nothing. That was the most dangerous sentence in the file — an unset `DEMO_MODE` is now the *first* thing to check rather than something to rule out. The line now says it moved and why. The crash-loop example was the PayPal triple, which cannot occur while `DEMO_MODE` is `true`. The message an operator will actually see during the demo interim — `DEMO_MODE is required and must be exactly 'true' or 'false'` — appeared nowhere in the runbook. Both forms are shown now, in the order they are likely to be hit. Added what neither document said: Compose only *warns* about an unset variable and deploys anyway. In Portainer's stack UI that warning is easy to miss, and the container then crash-loops under `restart: unless-stopped` — loud in the log, invisible in a glance at the stack list. The container log is the signal, not the deploy output. The compose header carried two claims that #190 falsified and did not correct: that the PayPal secrets are required "because DEMO_MODE is false below", and that only secrets are interpolated. Both now describe the file as it is. This is the drift `composeEnvironment.test.ts` exists to prevent, surfacing in the one place no test can reach — the guard keeps the compose file honest about its own intent and cannot see the runbook beside it. Verified: 278 backend unit tests pass, including the compose guard that parses this file, and a sweep for the stale claims finds none left. Closes #196 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
bermudalamb
|
6104ebb459 |
feat(ops): read DEMO_MODE from the stack rather than the compose file (#190)
Reverses the position the previous commit took. Hardcoding it made a value that gets flipped without a code change require a commit and a merge to flip, which is backwards — and it is the operator's call, not the file's.
No default, deliberately. `${DEMO_MODE:-false}` is the obvious form and the wrong one: a default decides whether the shop takes money on the operator's behalf, silently, whichever way it points. Having none is safe rather than fragile because `checkDemoMode` is strict — an unset stack variable substitutes to an empty string, and anything that is not exactly `true` or `false` refuses to boot naming DEMO_MODE. That strictness is the whole reason interpolating this one is defensible, so the line says so.
The compose guard had to learn the difference. It hands each deploying file's entries to the real validator, and a literal `${DEMO_MODE}` is not a value `checkDemoMode` accepts, so both the DEMO_MODE assertion and the validateEnv check failed the moment the file stopped holding a literal. Each deployment now declares the stack variables it supplies, and a bare `${VAR}` named there resolves to the declared value before the file is validated. Every other bare `${VAR}` stays opaque exactly as before — those are secrets, and what is checked of them is that the line exists.
Be clear about what that guard can prove. It cannot see Portainer, so it does not verify the stack actually holds `true`; nothing in this repository can. What it does is keep the intent beside the file and make the pair inseparable — hardcode the compose line and the registry disagrees, change the registry and it no longer describes the file. The runtime half is the boot check, which fails loudly rather than falling back. Verified by mutation: hardcoding `false` fails the DEMO_MODE assertion, and deleting the line fails that and `validateEnv`.
Restoring real payments is now two Portainer values and a redeploy, with no commit — which is what #190 asks for.
|
||
|
bermudalamb
|
12b2f09d79 |
docs(ops): point the demo-mode notes at the right issue (#190)
The tracking issue was filed as #190; the compose banner and the guard test both said #191, guessing the number before it existed. A note that points at the wrong issue is worse than no note when the thing it tracks is production not taking money. |
||
|
bermudalamb
|
0a830cad1f |
feat(ops): put production in demo mode to complete the cutover (#191)
Production could not boot during the cutover to the committed compose file: `DEMO_MODE` is false there, which makes the three PayPal secrets required, and they were not available. Demo mode is the interim the compose file's own header sanctions for exactly this — the whole cart and checkout flow works and nobody is ever charged.
Two things made this cost more than it should have, and both are now written down rather than left to be rediscovered.
`DEMO_MODE` is hardcoded rather than interpolated, so setting a `DEMO_MODE` stack variable in Portainer does nothing at all — there is no `${...}` for it to substitute into and the file's value wins silently. That hardcoding is right: the one value deciding whether the shop takes money should not be flippable from a web UI without a commit anybody can read. But the failure mode reads as "I set it and it ignored me", so the line now says so.
Declaring `PAYPAL_CLIENT_ID=` with an empty value is identical to not declaring it. `isPresent` rejects a blank string deliberately, because set-to-nothing is a mistake rather than a value.
The state is loud in both places that can see it. The compose file leads with a banner saying production is taking no money, and `composeEnvironment.test.ts` asserts `DEMO_MODE` is `true` — that assertion is the guard, not a formality: it fails the moment the file and the expectation disagree, in either direction, so this cannot be switched back quietly and cannot be left on unnoticed.
#191 restores it.
|
||
|
bermudalamb
|
cf9003bc2d |
docs(ops): write down how to cut production over to the committed compose file (#175)
#118 put `docker-compose.prod.yml` in the repository and argued at length for why it belongs there. It never produced the procedure for getting production from where it is — a stack living in Portainer's web editor — to where that file expects it to be. What existed was scattered and none of it was a procedure. The compose header described the destination: name the stack this, deploy it as a git repository stack with these settings, rename any variables whose names differ. The README's deployment section is a good runbook for the routine redeploy, and assumes the stack is already in the right form. So the riskiest deployment operation this project has was the one with no steps written down. `docs/ops/production-stack-cutover.md` is those steps, in the order that matters, with the two facts that decide whether the operation is safe stated up front rather than left to be inferred from the volumes block. The first is that all persistent data is bind-mounted from the NAS filesystem rather than held in Docker-managed volumes, so deleting the stack cannot lose the database or the product images. That is what makes the cutover recoverable at all, and step 2 verifies it rather than trusting it — the `docker inspect` there discriminates `bind` from `volume` and says to stop on `volume`, because this runbook does not cover that case. The second is that both services set an explicit `container_name`, so the old containers must be gone before the new stack starts or the deploy fails on a collision that reads like a Portainer bug rather than a sequencing mistake. It also captures what is simply lost if not recorded first. Portainer stack variables belong to the stack and are discarded with it; they are all secrets, and a stack brought back up with a different `DB_PASSWORD` than the data directory was initialised with cannot authenticate against its own database. Recording them is step 2 and it is what makes the rollback credible. The verification section separates the boot warnings that are correct in production — no mail allowlist, no uploads origin yet — from the one that means a variable never arrived. That distinction is the whole failure mode #118 was about: a stack variable whose name matches nothing in the file is substituted nowhere and never reaches the container, and the failure reads as "I set it and it says it is not set". Every diagnostic command in it was run rather than written from memory. The mount inspection was checked against a container that genuinely uses a named volume, to confirm it reports the difference the step depends on. The compose header now points at the runbook for the procedure and keeps the rationale that belongs at the point of use — why there is no `build:`, why most values are hardcoded, why `NODE_ENV` is absent. The README says plainly that its section is the routine deploy and links to the other one. Closes #175 |
||
|
bermudalamb
|
cf1680dbfb |
feat(security): stop the app origin serving anything it does not recognise, and make the uploads origin configurable (#103)
The uploads directory is the only place in this application where content someone else authored is served over HTTP. #95 stopped a dangerous file being stored; this stops a stored file doing damage if one ever gets there anyway — through a gap, a path added later, a restore, or a file written before that validation existed. Two halves, complementary rather than alternative. The app's own origin now serves uploads defensively. An allowlist of the three extensions the upload path can produce, so a `.html` or a `.svg` on disk is simply not a file this application hands out — 404, the same answer as a file that is not there, so the response cannot be used to learn which paths exist. An allowlist rather than a denylist because a denylist has to anticipate every type a browser might execute, which is a moving target across browsers and years, while this only has to know three. The content type is stated explicitly from that same list rather than sniffed or guessed from a name someone else chose, paired with `nosniff`. `default-src 'none'; sandbox` gives a directly-navigated file no capabilities at all, which is the only way one of these can do harm — an `<img>` embed does not execute script. Writes get 405 rather than falling through to a 404 that suggests the path is wrong. The other half is the separate origin, which is the real fix, because the origin is the whole unit of trust in a browser. That needs a hostname and a certificate, which live outside this repository, so what is here is the switch: `UPLOADS_BASE_URL`, sent to the frontend at runtime through `/api/config` and joined onto stored paths by `uploadUrl`. Empty means the app's own origin, which is the default and what local development has, so nothing changes until it is pointed somewhere. Stored paths stay site-relative. A stored value outlives any hostname baked into it, and rewriting them would be a migration to undo the day the hostname changes. Runtime rather than built in, so one image serves every environment — the same reason `paypalClientId` and `demoMode` are already there. `UPLOADS_BASE_URL` has a line in `docker-compose.prod.yml` while still empty, deliberately: a Portainer stack variable with no line there is substituted into the file and never reaches the container, which is exactly how `UPLOADS_DIR` went missing on 2026-08-23. Unset warns at boot, in the same shape as the admin gate — a working configuration with one defence switched off is worth saying out loud. Set without a scheme is refused outright, because a bare hostname joins onto a stored path as if it were relative and breaks every image on the site rather than failing visibly. The compose guard now resolves `${VAR:-default}` to its default, which is what the container actually receives when the stack variable behind it is unset. A bare `${VAR}` is still left opaque, so a required variable referenced that way goes on counting as present — that check is about the line existing, not about the stack being filled in. Closes #103 |
||
|
bermudalamb
|
ebefcbb76b |
feat(ops): schedule database and uploads backups, and document the restore (#147)
The only copy of every customer, order and one-of-a-kind item was the live Postgres data directory, plus whatever the deploy checklist's manual pg_dump happened to have caught. That dump is good and stays, but it only runs when someone deploys: a quiet week meant the newest copy of real customer data was a week old, and nothing bounded the gap. Two services rather than one, because they are different jobs. The database is small, changes constantly and wants a logical dump — daily, gzipped, 7/4/6 daily-weekly-monthly retention. Uploads are large and append-mostly and want an archive — weekly, 56 days, mounted read-only so a backup process cannot damage the thing it is backing up. Forcing both through one tool would serve one of them badly. The dumper is pinned to postgres-backup-local:16 to match the server. pg_dump refuses to dump a server newer than itself, so a floating tag is a backup that stops working the day Postgres is upgraded — silently, because nothing reads a dump until it is needed. It depends_on the database's existing pg_isready healthcheck, which is the constraint that shaped this: a dumper is a client, and without that the first run after a NAS reboot races Postgres coming up. Both carry a staleness healthcheck rather than trusting the schedule. A regime that stopped a month ago is indistinguishable from a working one until a restore is attempted, and `find -mmin` is the cheapest thing that tells them apart. It surfaces in Portainer beside the app rather than somewhere separate to remember to look. Windows are the interval plus grace — 26 hours daily, 9 days weekly — so a late run is not a failure, and start_period covers the first cycle when nothing has been written yet. Verified rather than assumed. Both images were pulled and checked to have a shell and `find`, since a CMD-SHELL healthcheck against an image without one reports unhealthy forever. postgres-backup-local:16 ships pg_dump 16.10 against the postgres:16 server. The healthcheck expression was exercised three ways in the image itself — empty directory, fresh artifact, and one aged three days — and returns unhealthy, healthy, unhealthy. The compose file parses and the #118 drift guard still passes over it. Three things these deliberately do not cover, written into the compose file and the doc rather than left to be discovered: They run while the stack runs, so they cannot protect the stack's own teardown. Deleting the Portainer stack deletes them too. That is why the deploy checklist's manual dump stays, and README now says so where the checklist is. They write to the same volume as the data they protect. That survives a bad migration, a dropped table, a bad deploy and a stack deletion, and not the disk. Getting a copy off /volume1 is a Synology-side job and is what turns this from a convenience into a guarantee. Daily database against weekly uploads leaves a window where a restore pairs the two from different moments. An orphaned image is harmless; a row without its image is a broken thumbnail on one recent item, usually still on the admin's machine. Neither is data loss, and a synchronised snapshot is not worth the complexity to avoid it. The restore procedure leads with practising on a throwaway database, because an untested backup is a file of unknown validity and a truncated dump looks exactly like a good one until it matters. It checks row counts and the pgmigrations head — the migration check being the one most easily skipped and most likely to bite, since a dump older than the code restores a schema the app will fail against. Both open items are listed as unticked in the doc: no off-volume copy exists yet, and no restore has been performed. Until the second is done this documents an untested procedure, and it says so. Refs #147 |
||
|
bermudalamb
|
e48c7f585b |
fix(deploy): run the image QA reviewed rather than rebuilding production (#146)
The production compose committed in #145 carried `build:` and `pull_policy: build`, copied from the QA stack without thinking about what they mean there. QA builds from this repository because QA is where a change is first assembled and reviewed. Production is not that; production is where the reviewed thing runs. Worse, it contradicted the deploy this repository already documents. README's production steps promote the exact image QA reviewed — `docker tag redefined-designs:qa redefined-designs:latest` — with the stated reason that what ships is what was tested. A compose file that rebuilds instead quietly overrode that, and the two would have disagreed at the moment it mattered. Rebuilding would be a defensible shortcut if a rebuild of the same commit produced the same image. It does not. The Dockerfile copies package.json without package-lock.json and installs with `npm install`, so both lockfiles in this repository are ignored in every build stage and every dependency range is resolved afresh. Two builds of one commit, minutes apart, can differ in any transitive dependency that published in between. "Same git ref" is therefore not "same image", and the reviewed bytes are the only thing that is. So the service now declares `image: redefined-designs:latest` and nothing else. The image has to exist before the stack starts; a first deploy or a pruned NAS fails with "image not found" rather than silently building something new. That is the intended behaviour and is written into the file rather than left to be discovered. The header records what removing `pull_policy: build` costs, because it is not free. That option exists in QA to stop a redeploy reusing a stale tag and appearing to succeed while running old code. Production reintroduces the same risk by a different route — a redeploy that reuses the previous `latest` because nobody re-tagged — so the promotion is load-bearing rather than a convenience, and the deploy has to say which image it is promoting. Also corrects the README's claim that production runs from a stack outside this repository and so cannot be checked. That was true when it was written and stopped being true in #118; leaving it would have taught the next reader that the environment which just failed to boot is the one nothing watches. The remaining half of #146 — copying the lockfiles and installing with `npm ci` so any rebuild means something — is not in this commit. It changes what CI and QA build as well as production, and belongs with its own verification that an unchanged commit produces an unchanged dependency tree. Refs #146 |
||
|
bermudalamb
|
3fde6fc6bf |
fix(deploy): commit production's compose and bring it under the drift guard (#118)
Production refused to boot with "UPLOADS_DIR is required and is not set" while UPLOADS_DIR was set in Portainer's stack variables. Both statements were true at once. Portainer substitutes stack variables into the compose file rather than handing them to the container, so a variable with no line in the file never reaches the app — behaviour the QA compose already warns about at its ADMIN_GATE_SECRET entry, hit in production where nothing was watching for it.
Nothing could have caught it. The drift guard reads docker-compose.qa.yml, and production ran from a Portainer stack outside the repository that no test could see. That is worse than an even gap: UPLOADS_DIR is in ALWAYS_REQUIRED and the test was green, so the natural reading was that the deploying environments set it. QA did. Production did not.
So production's compose is now a file in the repository, deployed as a git repository stack rather than pasted into the web editor — otherwise the committed copy and the running copy drift apart again, which is the whole problem.
Values are hardcoded rather than interpolated wherever they are not secrets. Only a secret has a reason to stay out of the repository, and every interpolation is another chance for the failure above. UPLOADS_DIR in particular has to agree with the volume mapping, and splitting it across two files is how they drift.
The guard now runs over every deployment rather than QA alone, and checks each by handing its parsed entries to validateEnv itself rather than restating the rules. A restatement is one more copy to drift; running the real validator means the file is checked against exactly what the container checks at boot. Interpolated ${SECRET} values count as present, which is right — what is being guarded is that the line exists, since that is what decides whether the value reaches the container.
Environments differ on purpose, so the expectations are registered per file rather than shared: QA is demo mode with a mail allowlist and no PayPal credentials, production is the reverse of all three. A root-level compose file that is not registered fails the last test, so adding an environment forces the decision instead of silently inheriting whatever the loop asserted.
Verified by removing the UPLOADS_DIR line from the production file and confirming three tests fail, one of them reproducing the exact boot error. A guard of this kind that has never been seen to fire is indistinguishable from one that cannot.
Two things found while writing this and deliberately not changed here. RESERVATION_MINUTES is set in QA's compose and read nowhere in the code — drift in the opposite direction, which #118's scan half should catch. And production publishes 32750 on every interface exactly as QA does; that is #117, and the reasoning is recorded in a comment at the ports block rather than acted on, since changing it needs the proxy host entry repointed in the same pass.
Refs #118
|