docs: update project context with deployment lessons and cart architecture

.claude/project-context.md

@@ -1,12 +1,17 @@
 # Redefined Designs — Project Context
 
+## Environment
+
+This is a **self-hosted** application running on a **Synology NAS**, deployed and managed via **Portainer** (Docker Compose stacks through Portainer's UI, not raw `docker compose` CLI on the host). This is a homelab project, not a cloud-hosted SaaS — deployment is manual, there's no CI/CD pipeline to production, and infrastructure decisions are shaped by NAS constraints (Synology's permission model, limited CPU/RAM, no exposed Postgres port, etc.).
+
+**Repo**: `https://gitea.bermudalamb.synology.me/bermudalamb/redefined-designs` (self-hosted Gitea, also on the NAS)
+**Production URL**: `https://redefined-designs.bermudalamb.synology.me`
+**Container names**: `redefined-designs-syn` (app), `redefined-designs-db-syn` (Postgres)
+**NAS repo path**: `/volume1/docker/redefined-designs`
+
 ## What this is
 
-A storefront for one-of-a-kind items (each item has quantity 1 — once sold, it's gone). Built solo, deployed to a personal Synology NAS homelab via Docker/Portainer, behind Nginx Proxy Manager + authentik SSO for the admin panel.
-
-**Repo**: `https://gitea.bermudalamb.synology.me/bermudalamb/redefined-designs`
-**Production URL**: `https://redefined-designs.bermudalamb.synology.me`
-**Local dev**: see `README.md`
+A storefront for one-of-a-kind items (quantity 1 per item — once sold, it's gone). Public storefront + cart + checkout, with an authentik-SSO-gated admin panel for inventory/customer/settings management.
 
 ## Tech stack
 
@@ -14,51 +19,150 @@ A storefront for one-of-a-kind items (each item has quantity 1 — once sold, it
 - **Frontend**: React + TypeScript + antd (Vite build), served as static files by the backend in production
 - **Auth**: two separate systems —
   - **Customers**: email/password, bcrypt, session cookie (`rd_session`), custom-built (not authentik)
-  - **Admin**: authentik SSO via NPM forward-auth, gated only on `/admin` and `/api/admin/*` — the storefront itself is public
-- **Payments**: PayPal Orders v2 API (multi-item cart checkout, single transaction)
-- **Shipping validation**: USPS Addresses API (OAuth2/REST, not the deprecated Web Tools XML API) — optional, degrades gracefully if unconfigured
+  - **Admin**: authentik SSO via Nginx Proxy Manager forward-auth, gated only on `/admin` and `/api/admin/*` — the storefront itself is public
+- **Payments**: PayPal Orders v2 API — real multi-item cart checkout, single transaction, itemized breakdown
+- **Shipping validation**: USPS Addresses API (OAuth2/REST — `apis.usps.com`, not the deprecated Web Tools XML API), US-only, optional/graceful if unconfigured
 - **Email**: nodemailer via Gmail SMTP — optional, degrades gracefully if unconfigured (logs a warning, skips sending)
-- **Testing**: Jest (unit + integration against a disposable tmpfs Postgres), Playwright (e2e)
-- **CI**: Gitea Actions — two workflows, `sonarqube.yml` (static analysis + TS build check) and `tests.yml` (unit/integration/e2e with job summaries)
+- **Scheduled jobs**: `node-cron` in-process (cart expiry sweep every 5 min via `setInterval`, daily cart-reminder emails at 9am via cron)
+- **DB migrations**: `node-pg-migrate` — NOT Entity Framework (this is a Node/TS backend, EF doesn't apply here). Migration files live in `backend/migrations/`, run via `node migrate.js up` (bundled into the production image specifically so it can be invoked via `docker exec` against the live container). There's no `init.sql` anymore — it was deleted when migrations were introduced; the migration files are the single source of schema truth.
+- **Testing**: Jest (unit + integration against a disposable tmpfs Postgres via `node-pg-migrate`), Playwright (e2e)
+- **CI**: Gitea Actions — `sonarqube.yml` (static analysis + TS build check) and `tests.yml` (unit/integration/e2e with job summaries posted to Gitea's step summary)
 
 ## Data model (key tables)
 
 - `items` — one-of-a-kind inventory. `status`: `available | reserved | sold`
 - `item_images` — multiple images per item (front/back/etc.), ordered
 - `customers` — accounts, with `marketing_consent` (explicit opt-in, GDPR-style — unchecked by default, timestamped, unsubscribe token)
-- `carts` / `cart_items` — one cart per customer, items reserved with an admin-configurable expiry (`admin_settings.cart_expiry_hours`, default 24h)
-- `shipping_addresses` — per-customer, USPS-validated when configured
+- `admin_settings` — key/value store, currently just `cart_expiry_hours` (admin-configurable, default 24)
+- `carts` / `cart_items` — one cart per customer, items reserved with the admin-configurable expiry; unique constraint on `item_id` means an item can only be in one cart at a time
+- `shipping_addresses` — per-customer, USPS-validated when USPS creds are configured
 - `checkouts` / `checkout_items` — one row per multi-item PayPal/demo transaction, snapshotting item+price at purchase time
 - `orders` — per-item purchase records, linked to `checkout_id` and `customer_id`
 
 ## Key architectural decisions
 
-1. **Cart replaced the old single-item "Buy Now" flow entirely.** Old `routes/paypal.ts` / `routes/demo.ts` (single-item) exist on disk but are unmounted — superseded by `routes/cartCheckout.ts` which does real multi-item PayPal orders (one `purchase_unit` with an itemized `items[]` breakdown, single transaction, single capture).
-2. **One-of-a-kind + double-sell prevention**: adding to cart immediately flips `item.status` to `reserved` inside a DB transaction with `FOR UPDATE` locking — no two customers can hold the same item. Expired cart holds are swept every 5 minutes back to `available`.
-3. **Demo mode** (`DEMO_MODE=true` env var): lets the whole site — including checkout — work with zero PayPal/USPS credentials configured, useful for local dev and just generally having a working fallback. This "degrade gracefully when a third-party integration isn't configured" pattern is used consistently: PayPal, USPS, SMTP all no-op safely if unset rather than crashing.
-4. **GDPR/consent handling**: marketing consent is never pre-checked, is timestamped with the exact consent text shown, and customers have working self-service data export + account deletion endpoints (`/api/customers/me/export`, `DELETE /api/customers/me`).
-5. **Admin panel security boundary**: NPM's Advanced nginx config only wraps `auth_request` around `location ~ ^/(admin|api/admin)` — everything else (storefront, cart, checkout, PayPal webhook) bypasses authentik entirely. This is a deliberate split, not an oversight — don't accidentally widen or narrow that regex without checking both directions.
-6. **Cookie `secure` flag** is gated on `NODE_ENV === 'production'`, not hardcoded `true` — otherwise integration tests (plain HTTP, no TLS) silently fail to persist sessions. Learned this the hard way once already.
+1. **Cart replaced the old single-item "Buy Now" flow entirely.** Items are added to cart (immediately flips `item.status` to `reserved` inside a DB transaction with `FOR UPDATE` locking — no double-sell race), then checked out together in one real multi-item PayPal transaction (one `purchase_unit`, itemized `items[]` breakdown, single capture). Old single-item route files (`routes/paypal.ts`, `routes/demo.ts`) still exist on disk but are unmounted from `app.ts` — safe to delete, never got cleaned up.
+2. **Demo mode** (`DEMO_MODE=true` env var): lets the whole site — including full cart checkout — work with zero PayPal/USPS credentials configured. Same "degrade gracefully when a third-party integration isn't configured" pattern applies to PayPal, USPS, and SMTP — none of them crash the app if unset, they just no-op or skip.
+3. **Account creation is prompted at add-to-cart**, not gated earlier — clicking "Add to Cart" while logged out opens an inline modal (register or login tabs), and on success the item is added automatically.
+4. **GDPR/consent handling**: marketing consent is never pre-checked, is timestamped with the exact consent text shown, and customers have working self-service data export + account deletion endpoints.
+5. **Admin panel security boundary**: NPM's Advanced nginx config only wraps `auth_request` around `location ~ ^/(admin|api/admin)` — everything else (storefront, cart, checkout, PayPal webhook) bypasses authentik entirely. Deliberate split — don't accidentally widen or narrow that regex without checking both directions.
+6. **Cookie `secure` flag** is gated on `NODE_ENV === 'production'`, not hardcoded `true` — otherwise integration tests (plain HTTP, no TLS) silently fail to persist sessions.
+7. **`app.ts`/`server.ts` split**: `app.ts` exports the Express app with no `.listen()` call, so tests can import and exercise it via `supertest` without binding a port. `server.ts` is the actual entry point — imports `app`, adds the cart-expiry sweep and cron job, calls `.listen()`. Any new background job or process-level concern goes in `server.ts`, not `app.ts`.
 
-## Conventions (apply to all future work on this repo)
+## Deployment workflow — read this before touching the NAS
+
+This is where nearly every real time-sink in this project's history happened. The process:
+
+1. Edit files locally in **VS Code**, on a branch, using **PowerShell** `git` (not SSH).
+2. Push, open a PR in Gitea, merge into `main`.
+3. SSH into the NAS, pull the repo, rebuild the Docker image, redeploy the Portainer stack.
+
+### Hard-won lessons from this project's history — internalize these
+
+- **The NAS's local git clone does NOT have a real `git` binary.** DSM doesn't ship one. Git operations on the NAS go through a throwaway `alpine/git` Docker container, wrapped in a shell function:
+```bash
+  gitc() {
+    sudo docker run --rm -it \
+      -v /volume1/docker/redefined-designs:/repo \
+      -v ~/.gitconfig-docker/.gitconfig:/root/.gitconfig \
+      -w /repo \
+      alpine/git "$@"
+  }
+```
+  **This function must be redefined every new SSH session** — it only lives in shell memory, doesn't persist to `~/.bashrc` (never got added there). Always redefine it first thing after SSH'ing in, before running any `gitc` command.
+
+- **Git identity and `safe.directory` are NOT in the container image** — they're mounted from `~/.gitconfig-docker/.gitconfig` on the host, which is what makes them persist across `gitc` invocations (each `gitc` call is a brand-new container). If that file is ever missing, recreate it:
+
+[safe]
+directory = /repo
+[user]
+email = thomlamb@gmail.com
+name = Thom Lamb
+
+- **The NAS git working directory will show every file as "modified" even right after a clean pull, if `core.fileMode` isn't disabled.** Synology forces broad `-rwxrwxrwx+` permissions on all files regardless of what git committed, and git tracks the executable bit as part of a file's tracked "mode" by default — so every file looks dirty even when content is byte-identical. Fix once per repo:
+```bash
+  gitc config core.fileMode false
+```
+  This is a real trap: `git status` showing a long "modified" list after a pull does NOT necessarily mean there are real local edits — check `core.fileMode` before assuming a merge conflict or lost work.
+
+- **If `gitc pull` refuses with "local changes would be overwritten,"** don't try to cherry-pick which files to discard — on this project, that approach has repeatedly failed to fully resolve since the modified-file list is often long and partially spurious (see `core.fileMode` above). Just reset the NAS clone to match origin exactly, since the NAS is never the source of truth for code:
+```bash
+  gitc fetch origin
+  gitc reset --hard origin/main
+```
+
+- **Docker layer caching has caused multiple "the fix didn't work" false alarms.** After any code change, rebuild with `--no-cache` and actually watch the output — don't assume `docker build` picked up new files just because the command succeeded:
+```bash
+  sudo docker build --no-cache -t redefined-designs:latest /volume1/docker/redefined-designs
+```
+  Confirm it reaches `Successfully tagged redefined-designs:latest`. A build that errors partway through (e.g. a TypeScript compile error) leaves the OLD image running untouched — Docker doesn't warn loudly about this, it just silently keeps serving stale code.
+
+- **`docker exec` does not expand shell globs** (`*.js`, `*.sql`) unless you explicitly invoke a shell:
+```bash
+  # WRONG — "No such file or directory"
+  sudo docker exec redefined-designs-syn grep -l "text" /app/public/assets/*.js
+  # RIGHT
+  sudo docker exec redefined-designs-syn sh -c "grep -l 'text' /app/public/assets/*.js"
+```
+
+- **Verify a fix actually deployed before declaring victory** — the recurring failure pattern this project has hit repeatedly is: code is correct in the repo → but the branch was never actually merged, OR the merge happened but the NAS never pulled, OR the pull happened but the Docker image was never rebuilt, OR the rebuild happened but used a stale layer cache. Each of these produces the identical symptom ("I made the fix but the UI/behavior didn't change"). The reliable way to root-cause it, in order:
+  1. `git log main --oneline` (local/PowerShell) — is the commit actually merged?
+  2. `gitc log --oneline` (NAS) — did the NAS actually pull it?
+  3. `grep` the actual source file on the NAS disk for a distinguishing string from the fix
+  4. Rebuild with `--no-cache`, watch it complete
+  5. `docker exec` into the container and `grep` the **built output** (`/app/public/assets/*.js` for frontend, `/app/dist/*.js` for backend) for the same string — this is the only step that confirms the deployed artifact, not just the source, has the fix
+  6. Hard-refresh / incognito window in the browser (aggressive JS bundle caching on both desktop and mobile browsers has caused false "still broken" reports even after a correct deploy)
+
+- **Files must be at the repo root where the Dockerfile expects them.** The `Dockerfile` lives at the repo root (not `backend/Dockerfile` — that was a real, repeat point of confusion this project hit multiple times). `docker build ... /volume1/docker/redefined-designs` uses whatever `Dockerfile` sits directly in that directory.
+
+- **Always double check `git status` shows a file as staged before committing**, and check it again after committing to confirm a clean working tree before pushing. This project has repeatedly had files silently left out of a commit (most notably `Admin.tsx` once, and the entire cart feature's branch/push never happening at all despite believing it had) — the cost of that mistake compounds badly (hours of debugging "why doesn't the deployed code match what I was shown"), so the few seconds to verify `git status` at each step is well worth it.
+
+### Standard deploy sequence, once code is confirmed on `main`
+
+```bash
+cd /volume1/docker/redefined-designs
+gitc() { sudo docker run --rm -it -v /volume1/docker/redefined-designs:/repo -v ~/.gitconfig-docker/.gitconfig:/root/.gitconfig -w /repo alpine/git "$@"; }
+gitc pull origin main
+gitc log --oneline -3   # confirm the expected commit is at the top
+sudo docker build --no-cache -t redefined-designs:latest /volume1/docker/redefined-designs
+sudo docker stop redefined-designs-syn
+sudo docker rm redefined-designs-syn
+# redeploy the stack in Portainer UI
+```
+
+### Running migrations against the live DB (via Portainer-managed container)
+
+```bash
+sudo docker exec redefined-designs-syn node migrate.js up
+sudo docker exec -it redefined-designs-db-syn psql -U redefined -d redefined -c "SELECT name, run_on FROM pgmigrations ORDER BY run_on;"
+```
+
+## Conventions
 
 - **Every commit message follows [Conventional Commits](https://www.conventionalcommits.org/)**: `feat:`, `fix:`, `chore:`, `docs:`, `test:`, `ci:`, `refactor:`, etc.
 - **Never commit directly to `main`.** Always branch: `feature/<short-description>` or `fix/<short-description>`. Open a PR, merge, branch auto-deletes (repo setting is on).
-- File edits are made directly in VS Code (project is cloned locally) and pushed via PowerShell `git`, not the old NAS-SSH-with-Docker-wrapped-git workflow from earlier in this project's history (that was only used before the repo existed locally in VS Code).
-- Production deploys are manual: `git pull` on the NAS copy → `docker build --no-cache` → stop/rm the container → redeploy the Portainer stack. No CD pipeline exists yet.
+- Local dev/editing happens in **VS Code**, pushed via **PowerShell** `git` — the NAS-side `gitc` workflow is *only* for pulling already-merged code down to deploy, never for authoring changes.
+
+## Testing
+
+- **Unit tests**: `cd backend && npm run test:unit` — no DB required
+- **Integration tests**: `npm run db:test:up` (disposable tmpfs Postgres via `docker-compose.test.yml`) → `npm run migrate:up` → `npm run test:integration` → `npm run db:test:down`
+- **E2e (Playwright)**: needs backend running against a migrated DB; `cd frontend && npm run test:e2e`
+- CI (`tests.yml`) runs all three as separate jobs, each posting a pass/fail summary to Gitea's job Summary tab. The `frontend-e2e` job runs `node migrate.js up` against a Postgres service container — same migration mechanism as everywhere else, no schema duplication anywhere in the project anymore.
 
 ## Known gaps / natural next steps
 
-- **Tests don't cover the cart/checkout/shipping feature yet** — only the pre-cart backend surface has unit/integration tests. Worth adding before trusting this in front of real customers.
-- **No CD** — the Gitea Actions workflows run tests/analysis but don't deploy. Manual NAS rebuild is still required after every merge.
-- **SonarQube CI still uses the `admin` token**, not a dedicated `gitea-ci` user (that was flagged as a to-do on the original .NET project this pattern was copied from, never circled back to for this repo).
-- **USPS/PayPal credentials**: check whether these are actually populated in the Portainer stack env vars before assuming checkout works end-to-end in production — `DEMO_MODE` may still be the only thing actually exercised.
-- **Double opt-in for marketing email** was discussed and deliberately deferred (single opt-in + easy unsubscribe was judged sufficient for now) — revisit if EU customer volume grows.
-- **Daily cart reminder emails** fire via `node-cron` inside the app process at 9am container-local time — if the container restarts frequently or memory pressure causes crashes (a known NAS constraint per the broader homelab), reminders could silently stop firing with no alerting on that failure mode.
+- **No CD** — Gitea Actions runs tests/analysis but doesn't deploy. Manual NAS rebuild is still required after every merge.
+- **SonarQube CI still uses the `admin` token**, not a dedicated `gitea-ci` user (flagged early on, never circled back to).
+- **USPS/PayPal credentials**: verify these are actually populated in the Portainer stack env vars before assuming checkout works end-to-end in production — `DEMO_MODE` may still be the only thing actually exercised live.
+- **Double opt-in for marketing email** was deliberately deferred (single opt-in + easy unsubscribe judged sufficient for now).
+- **Daily cart reminder emails** fire via `node-cron` inside the app process at 9am container-local time — if the container restarts frequently, reminders could silently stop firing with no alerting on that failure mode.
+- **Old single-item checkout route files** (`backend/src/routes/paypal.ts`, `backend/src/routes/demo.ts`) are dead code, unmounted but never deleted — safe cleanup opportunity.
+- **`backend/src/routes/shippingAddresses.ts` USPS OAuth token format** was implemented against the current (2026) USPS Addresses API docs at time of writing, using a JSON-body `client_credentials` request — if USPS changes their API again, this is the first place to check.
 
 ## Where to look first for common tasks
 
-- Add/change a DB table → `backend/init.sql` (fresh deploys) **and** a one-off migration run manually via `docker exec ... psql` against the live DB (no migration framework — see `backend/migrations/` for the pattern used so far)
+- Add/change a DB table → new file in `backend/migrations/` via `npm run migrate:create -- descriptive-name`, fill in `pgm.sql(...)` for up/down
 - Change cart/checkout behavior → `backend/src/routes/cartCheckout.ts` (the whole reserve → checkout → complete lifecycle lives here)
 - Change admin-configurable settings → `admin_settings` table + `backend/src/routes/adminSettings.ts` + `frontend/src/admin/Settings.tsx`
-- NPM/authentik/DSM reverse-proxy config for this app → not in this repo; documented in the homelab's broader Claude Project knowledge base, not here
+- NPM/authentik/DSM reverse-proxy config for this app → not in this repo; documented in the broader homelab's Claude Project knowledge base, not here