fix: run migrations on boot and stop failures rendering as empty (#23)

The storefront showed no inventory after deploying the categories/tags
release. No data was lost: the code queried categories/item_tags/
items.category_id against a database where the migration had not been
run, and that failure was invisible at every layer.

Three changes, each addressing one layer:

Migrations now run at container start, so deployed code cannot be ahead
of the schema and the easily-forgotten manual `docker exec migrate.js
up` step disappears. migrate.js waits for Postgres to accept
connections first, since the NAS brings the DB container up slower than
the app, and still exits non-zero so a bad migration stops the
container rather than serving a half-migrated schema.

Express 4 does not forward a rejected async handler, and no error
middleware was mounted, so a failing query never responded at all. Async
routes are now wrapped and an error middleware guarantees a 500. A hung
request is indistinguishable from an empty result in the UI, which is
how a schema mismatch came to read as "the store has no items".

The storefront now separates "request failed" from "no items" and offers
a retry. fetchItems/fetchFilterOptions throw on a non-OK response rather
than returning the parsed error body, which would have been set as the
item list and crashed the grid on .map.

Also restores the project-context update from 7fb5764, which was left
out of PR #24 and ended up dangling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.claude/project-context.md

@@ -11,7 +11,7 @@ This is a **self-hosted** application running on a **Synology NAS**, deployed an
 
 ## What this is
 
-A storefront for one-of-a-kind items (quantity 1 per item — once sold, it's gone). Public storefront + cart + checkout, with an authentik-SSO-gated admin panel for inventory/customer/settings management.
+A storefront for one-of-a-kind items (quantity 1 per item — once sold, it's gone). Public storefront + cart + checkout, with an authentik-SSO-gated admin panel for inventory, category/tag taxonomy, customer, and settings management. The storefront filters items by category, tags, and price range.
 
 ## Tech stack
 
@@ -27,11 +27,18 @@ A storefront for one-of-a-kind items (quantity 1 per item — once sold, it's go
 - **DB migrations**: `node-pg-migrate` — NOT Entity Framework (this is a Node/TS backend, EF doesn't apply here). Migration files live in `backend/migrations/`, run via `node migrate.js up` (bundled into the production image specifically so it can be invoked via `docker exec` against the live container). There's no `init.sql` anymore — it was deleted when migrations were introduced; the migration files are the single source of schema truth.
 - **Testing**: Jest (unit + integration against a disposable tmpfs Postgres via `node-pg-migrate`), Playwright (e2e)
 - **CI**: Gitea Actions — `sonarqube.yml` (static analysis + TS build check) and `tests.yml` (unit/integration/e2e with job summaries posted to Gitea's step summary)
+- **Node**: 20+ required. **Node 18 cannot run this project's tooling** — `node-pg-migrate` pulls in a `lru-cache` that calls `diagnostics_channel.tracingChannel()`, which doesn't exist before Node 19.9, so `npm run migrate:up` dies with `TypeError: (0 , U.tracingChannel) is not a function`. This is a confusing failure because it looks like a broken dependency rather than a version problem. The dev machine has nvm4w with both 18.16.1 and 24.13.1 installed and **18 is the active default**, so a command can be run against the newer binary without switching globally:
+```bash
+  export PATH="/c/Users/tlamb/AppData/Local/nvm/v24.13.1:$PATH"
+```
 
 ## Data model (key tables)
 
-- `items` — one-of-a-kind inventory. `status`: `available | reserved | sold`
+- `items` — one-of-a-kind inventory. `status`: `available | reserved | sold`. `category_id` is nullable (`NULL` = Uncategorized) and `ON DELETE SET NULL`
 - `item_images` — multiple images per item (front/back/etc.), ordered
+- `categories` — self-referencing tree (`parent_id`, `ON DELETE CASCADE`), arbitrary depth, one category per item. Two *partial* unique indexes enforce "siblings can't share a name" — a single plain unique constraint doesn't work, because a `NULL` `parent_id` compares unequal to every other `NULL` and duplicate root categories slip straight through
+- `tags` — flexible labels with a `color`; unique on `lower(name)`
+- `item_tags` — many-to-many between items and tags
 - `customers` — accounts, with `marketing_consent` (explicit opt-in, GDPR-style — unchecked by default, timestamped, unsubscribe token)
 - `admin_settings` — key/value store, currently just `cart_expiry_hours` (admin-configurable, default 24)
 - `carts` / `cart_items` — one cart per customer, items reserved with the admin-configurable expiry; unique constraint on `item_id` means an item can only be in one cart at a time
@@ -48,6 +55,20 @@ A storefront for one-of-a-kind items (quantity 1 per item — once sold, it's go
 5. **Admin panel security boundary**: NPM's Advanced nginx config only wraps `auth_request` around `location ~ ^/(admin|api/admin)` — everything else (storefront, cart, checkout, PayPal webhook) bypasses authentik entirely. Deliberate split — don't accidentally widen or narrow that regex without checking both directions.
 6. **Cookie `secure` flag** is gated on `NODE_ENV === 'production'`, not hardcoded `true` — otherwise integration tests (plain HTTP, no TLS) silently fail to persist sessions.
 7. **`app.ts`/`server.ts` split**: `app.ts` exports the Express app with no `.listen()` call, so tests can import and exercise it via `supertest` without binding a port. `server.ts` is the actual entry point — imports `app`, adds the cart-expiry sweep and cron job, calls `.listen()`. Any new background job or process-level concern goes in `server.ts`, not `app.ts`.
+8. **Categories are manual metadata, deliberately not a rule engine.** Issue #23's wording ("rules that dictate how the app automatically organizes items") was explicitly resolved with the author to mean an admin-built tree with a per-item assignment — there is no predicate evaluation anywhere, and adding one would be a new feature, not a completion of this one. Categories are single-valued per item on purpose, to stay distinct from multi-valued tags. **Tag filtering is AND** ("must have all"), not OR.
+9. **Item filtering happens in SQL, and malformed filter params return `400`** rather than being ignored — a broken filter link should show itself instead of quietly returning the whole catalogue. Parsing/SQL-building live in `backend/src/itemFilters.ts`, apart from the route so they're unit-testable with no database. Category filtering matches a node *and all descendants* via a recursive CTE; a materialized path column was rejected because reparenting would then have to rewrite every descendant's path, which is a standing drift risk.
+10. **Migrations run automatically at container start**, via `CMD ["sh", "-c", "node migrate.js up && node dist/server.js"]` in the `Dockerfile`. Deployed code can therefore never be ahead of the schema. This replaced a separate manual `docker exec ... node migrate.js up` step that was easy to forget — and forgetting it took the storefront down (see the incident note below). `migrate.js` waits for Postgres to accept connections before running (the NAS routinely brings the DB container up slower than the app), and exits non-zero on failure, so `&&` stops a bad migration from serving against a half-migrated schema.
+11. **Every async route is wrapped in `asyncRoute()` and the app mounts error middleware.** Express 4 does *not* forward a rejected promise from an async handler, and with no error middleware such a request **never responds at all** — it hangs until the client gives up. A hung request is indistinguishable from an empty result in the UI. New async routes must use the wrapper (`backend/src/asyncRoute.ts`); it becomes unnecessary only if the project moves to Express 5. Note the older route files predate this and are still unwrapped.
+12. **Item SELECTs aggregate with scalar subqueries, never `LEFT JOIN` + `GROUP BY`** — see `backend/src/itemSelect.ts`, the single source for both the public and admin shapes. Joining two one-to-many relations in one query multiplies their rows together: with the old shape, an item with 2 images and 3 tags repeated every image three times. This bit once already when tags were added. If a third one-to-many relation is ever attached to items, extend `itemSelect.ts` the same way rather than adding a join.
+
+## Frontend gotchas
+
+- **antd's `defaultExpandAll` on `Tree` is evaluated once, at mount.** Any branch created *after* that renders collapsed, and its children become unreachable — in the admin Categories tab this meant adding a subcategory made it appear to vanish. Anywhere a tree outlives the data it displays, track `expandedKeys` in state instead and expand the parent when a node is added or moved. The storefront's filter drawer gets away with `defaultExpandAll` only because `destroyOnHidden` remounts it after the categories have loaded — that's a coincidence, not a pattern to copy.
+- **`destroyOnClose` is deprecated in antd 5.29** in favour of `destroyOnHidden`. Several older files (`Admin.tsx`, `Cart.tsx`, `AuthPromptModal.tsx`) still use the old prop and log a console warning; new code uses `destroyOnHidden`.
+- **A closed antd `Drawer` keeps its content in the DOM** unless `destroyOnHidden` is set, so duplicate control labels (two "Clear all" buttons, say) collide in Playwright's strict mode even when only one is on screen. The active-filter chip row is marked `role="group" aria-label="Active filters"` so its controls stay addressable independently of the drawer's.
+- **An antd `Button` with an icon does not have the plain text as its accessible name** — the icon contributes leading whitespace, so `getByRole('button', { name: /^Filters/ })` matches nothing while `/Filters/` and the exact string `'Filters'` both work. Prefer a substring regex for icon buttons.
+- **A failed request must never render as an empty result.** `App.tsx` tracks a `failed` flag separately from `loading`, and shows an error with a Retry rather than falling through to `Empty`. Saying "no items yet" when the server is broken hides an outage and reads to a customer as an empty shop — this is exactly how the 2026-08-17 incident presented. `fetchItems`/`fetchFilterOptions` throw on a non-OK response rather than returning the parsed error body, which would otherwise be set as the item list and crash the grid on `.map`. Covered by `tests/e2e/storefront-errors.spec.ts`.
+- **antd import style is inconsistent across the codebase.** New files use deep imports (`import Drawer from 'antd/lib/drawer'`) per the user's standing rule; most older files still use barrel imports from `'antd'`. Don't churn existing files just to convert them.
 
 ## Deployment workflow — read this before touching the NAS
 
@@ -117,6 +138,10 @@ name = Thom Lamb
 
 - **Always double check `git status` shows a file as staged before committing**, and check it again after committing to confirm a clean working tree before pushing. This project has repeatedly had files silently left out of a commit (most notably `Admin.tsx` once, and the entire cart feature's branch/push never happening at all despite believing it had) — the cost of that mistake compounds badly (hours of debugging "why doesn't the deployed code match what I was shown"), so the few seconds to verify `git status` at each step is well worth it.
 
+- **Incident (2026-08-17): the storefront showed no inventory after a deploy.** No data was lost — the cause was deploying code that queried `categories` / `item_tags` / `items.category_id` against a database where that migration hadn't been run yet. Because Express 4 doesn't forward async errors and no error middleware existed, every item query **hung with no response**, and the frontend rendered that as an empty catalogue. Three changes now prevent this class: migrations run at container start (so code can't outrun the schema), an error middleware guarantees a response, and the storefront distinguishes "request failed" from "no items". **The general lesson is broader than migrations: any deploy where new code needs something the running environment doesn't have yet will surface as silence, not an error, unless something is written to make it loud.**
+
+- **Check `git branch --contains <sha>` before assuming a commit shipped.** A documentation commit made after the feature branch was pushed was left out of PR #24 entirely and ended up dangling — reachable only through the reflog. `git log` on the branch looked normal, because the missing commit simply wasn't there. If a commit was made after the push, confirm it actually made it onto the branch that got merged.
+
 ### Standard deploy sequence, once code is confirmed on `main`
 
 ```bash
@@ -128,20 +153,36 @@ sudo docker build --no-cache -t redefined-designs:latest /volume1/docker/redefin
 sudo docker stop redefined-designs-syn
 sudo docker rm redefined-designs-syn
 # redeploy the stack in Portainer UI
+# migrations now run automatically as the container starts — no separate step
+sudo docker logs redefined-designs-syn | head -20   # confirm migrations ran before the app came up
 ```
 
-### Running migrations against the live DB (via Portainer-managed container)
+### Migrations against the live DB
+
+**Migrations apply automatically at container start**, so a normal deploy needs no manual step. `docker logs` on the app container shows the migration output ahead of the server starting; a failed migration exits the container rather than serving a half-migrated schema.
+
+The manual invocation still works and is useful for applying a migration without redeploying, or for inspecting state:
 
 ```bash
 sudo docker exec redefined-designs-syn node migrate.js up
 sudo docker exec -it redefined-designs-db-syn psql -U redefined -d redefined -c "SELECT name, run_on FROM pgmigrations ORDER BY run_on;"
 ```
 
+If the storefront ever looks empty after a deploy, check the row count before assuming data loss — the far more likely cause is a schema/code mismatch:
+
+```bash
+sudo docker exec -it redefined-designs-db-syn psql -U redefined -d redefined -c "SELECT count(*) FROM items;"
+```
+
 ## Conventions
 
 - **Every commit message follows [Conventional Commits](https://www.conventionalcommits.org/)**: `feat:`, `fix:`, `chore:`, `docs:`, `test:`, `ci:`, `refactor:`, etc.
 - **Never commit directly to `main`.** Always branch: `feature/<short-description>` or `fix/<short-description>`. Open a PR, merge, branch auto-deletes (repo setting is on).
 - Local dev/editing happens in **VS Code**, pushed via **PowerShell** `git` — the NAS-side `gitc` workflow is *only* for pulling already-merged code down to deploy, never for authoring changes.
+- **Thom does the pushing.** Commit locally and stop; don't `git push` on his behalf.
+- **When work comes from a Gitea issue, post every clarifying question and its answer back to that issue as a comment** — including the options considered and why the rejected ones were rejected. The issue is the durable record; decisions made in a chat session are invisible to anyone reading it later. Post each round as the answers come in rather than batching everything to the end.
+- **Design specs live in `docs/superpowers/specs/YYYY-MM-DD-<topic>-design.md`** and are committed before implementation starts.
+- `.superpowers/` (scratch output from brainstorming tooling) is gitignored.
 
 ## Testing
 
@@ -150,6 +191,15 @@ sudo docker exec -it redefined-designs-db-syn psql -U redefined -d redefined -c
 - **E2e (Playwright)**: needs backend running against a migrated DB; `cd frontend && npm run test:e2e`
 - CI (`tests.yml`) runs all three as separate jobs, each posting a pass/fail summary to Gitea's job Summary tab. The `frontend-e2e` job runs `node migrate.js up` against a Postgres service container — same migration mechanism as everywhere else, no schema duplication anywhere in the project anymore.
 
+### E2E constraints — the local database is never reset
+
+Integration tests truncate between cases (`resetDb()` in `tests/integration/setup/testDb.ts` — **add any new table to that TRUNCATE list**, or state leaks between tests). Playwright specs have no such hook and run against whatever is already there, which locally accumulates across every previous run. Consequences worth knowing before writing a new spec:
+
+- **Name every fixture with a per-run suffix** and assert on those names only. A bare `.first()` or an unscoped `getByText` will find leftovers from an earlier run and produce a failure that looks like a product bug but isn't. This cost real debugging time.
+- **`beforeAll` runs once per worker, but a worker can be handed the same spec file in more than one batch**, re-running it against the module-cached suffix. `filters.spec.ts` therefore checks whether its fixtures already exist and returns early — without that, the second pass 409s on category names and duplicates every item, which then breaks strict-mode locators.
+- **Tables paginate.** With dozens of accumulated rows a freshly created record often isn't on page 1; `admin-taxonomy.spec.ts` confirms tag creation through the API rather than hunting for the row.
+- **Clicking a submit button only dispatches the request.** Wait for the resulting confirmation (`'Tag added'`) before querying the API, or the read races the write. This produced a one-in-four flake until fixed.
+
 ## Known gaps / natural next steps
 
 - **No CD** — Gitea Actions runs tests/analysis but doesn't deploy. Manual NAS rebuild is still required after every merge.
@@ -159,10 +209,19 @@ sudo docker exec -it redefined-designs-db-syn psql -U redefined -d redefined -c
 - **Daily cart reminder emails** fire via `node-cron` inside the app process at 9am container-local time — if the container restarts frequently, reminders could silently stop firing with no alerting on that failure mode.
 - **Old single-item checkout route files** (`backend/src/routes/paypal.ts`, `backend/src/routes/demo.ts`) are dead code, unmounted but never deleted — safe cleanup opportunity.
 - **`backend/src/routes/shippingAddresses.ts` USPS OAuth token format** was implemented against the current (2026) USPS Addresses API docs at time of writing, using a JSON-body `client_credentials` request — if USPS changes their API again, this is the first place to check.
+- **`TAG_COLORS` is duplicated** between `backend/src/utils.ts` and `frontend/src/admin/Tags.tsx`. The server validates against its copy, so editing one alone makes the admin colour picker offer values that get rejected with a `400`. There's no shared module between backend and frontend in this repo to put it in.
+- **The local e2e database accumulates junk indefinitely** — every Playwright run seeds categories, tags, and items that are never cleaned up, so the admin tables and filter drawer fill with `Furniture fmsxb…` noise over time. Harmless, but `npm run db:test:down` + `db:test:up` + `migrate:up` resets it when the clutter starts getting in the way. CI is unaffected (fresh service container per run).
+- **The storefront still lists sold items** and the price-range bounds are computed across all items regardless of status. Unchanged by the filters work — deliberately left as-is, but worth revisiting if sold stock ever outnumbers available stock.
+- **No admin-side filtering.** The admin inventory table shows category and tag columns but can't filter or search on them; with 100+ items that will start to hurt.
 
 ## Where to look first for common tasks
 
-- Add/change a DB table → new file in `backend/migrations/` via `npm run migrate:create -- descriptive-name`, fill in `pgm.sql(...)` for up/down
+- Add/change a DB table → new file in `backend/migrations/` via `npm run migrate:create -- descriptive-name`, fill in `pgm.sql(...)` for up/down, **and add the table to `resetDb()`** in `backend/tests/integration/setup/testDb.ts`
 - Change cart/checkout behavior → `backend/src/routes/cartCheckout.ts` (the whole reserve → checkout → complete lifecycle lives here)
 - Change admin-configurable settings → `admin_settings` table + `backend/src/routes/adminSettings.ts` + `frontend/src/admin/Settings.tsx`
+- Add a new async route → wrap the handler in `asyncRoute()` from `backend/src/asyncRoute.ts`, or a failure will hang the request instead of returning 500
+- Change what an item row returns → `backend/src/itemSelect.ts` (one place, used by both the public and admin routes)
+- Change storefront filtering → `backend/src/itemFilters.ts` (parsing + SQL), `backend/src/routes/filters.ts` (`/api/filters`, the drawer's single fetch), `frontend/src/filters.ts` (state, URL round-trip, tree building), `frontend/src/components/FilterDrawer.tsx`
+- Change category/tag management → `backend/src/routes/adminCategories.ts`, `backend/src/routes/adminTags.ts`, `frontend/src/admin/Categories.tsx`, `frontend/src/admin/Tags.tsx`
+- Change tag colours → `TAG_COLORS` + `tagColorFor()` in `backend/src/utils.ts`; the list is **duplicated** in `frontend/src/admin/Tags.tsx` for the override picker, and the server rejects anything outside it, so the two must be changed together
 - NPM/authentik/DSM reverse-proxy config for this app → not in this repo; documented in the broader homelab's Claude Project knowledge base, not here