redefined-designs/README.md

# Redefined Designs

One-of-a-kind item storefront. React + TypeScript + antd frontend (Vite), Express + TypeScript backend, Postgres, PayPal checkout, customer accounts with GDPR-style consent handling, and an authentik-gated admin panel.

## Project layout

backend/ Express + TypeScript API, Postgres access, PayPal integration
frontend/ React + TypeScript + antd storefront and admin UI (Vite)

## Prerequisites

- Node.js 20+
- Docker (for a local, disposable Postgres instance)
- npm

## Clone

```powershell
git clone https://gitea.bermudalamb.synology.me/bermudalamb/redefined-designs.git
cd redefined-designs
```

## Running locally

Commands below are shown for **PowerShell** (Windows). A bash equivalent is noted wherever the syntax differs.

### The short way

`scripts/start-local.ps1` does everything in this section — Postgres, migrations, the backend and the dev server — and `scripts/run-tests.ps1` runs the suites. The step-by-step instructions below are still accurate, and are what to reach for when something needs doing differently.

```powershell
.\scripts\start-local.ps1            # bring the whole stack up
.\scripts\start-local.ps1 -Fresh     # ...from an empty database
.\scripts\start-local.ps1 -Stop      # stop everything

.\scripts
un-tests.ps1 -Suite unit
.\scripts
un-tests.ps1 -Suite integration
.\scripts
un-tests.ps1 -Suite e2e
.\scripts
un-tests.ps1 -Suite all
```

Both scripts run `nvm use latest` first and verify the result is Node 20 or newer, then put the machine back to 18.16.1 when they finish — including when they fail partway, so an interrupted run does not leave the version switched. **`nvm use` rewrites a machine-global symlink, so this changes the Node version for every terminal on the machine while a script is running, not only the one you ran it in.** Both scripts say so as they do it.

The Node 20 floor is not arbitrary: `node-pg-migrate` pulls in an `lru-cache` that calls `diagnostics_channel.tracingChannel()`, which does not exist before Node 19.9. On Node 18 migrations die inside minified library code with `(0 , U.tracingChannel) is not a function`, which says nothing about versions.

`run-tests.ps1` brings up whatever a suite needs: the integration suite gets its own throwaway Postgres, started and stopped around the run (`-KeepTestDb` leaves it up, `-TestDbPort` moves it if the default is taken or Hyper-V has reserved it). The e2e suite needs the app stack, so start it with `start-local.ps1` first — the script checks and says so rather than letting every spec fail on a refused connection. `-Filter` passes through to the runner to select tests by file or name.

### 1. Start a local Postgres instance

The backend needs Postgres to talk to during local development. `backend/docker-compose.test.yml` spins up a throwaway, tmpfs-backed instance — no data persists between restarts, which is fine for local dev and tests.

```powershell
cd backend
npm run db:test:up
```

This starts Postgres on `localhost:55432`, database `redefined_test`.

### 2. Run migrations

**PowerShell / bash (same command, cross-platform via Node):**

cd backend
$env:PGHOST="localhost"; $env:PGPORT="55432"; $env:PGUSER="redefined_test"; $env:PGPASSWORD="redefined_test"; $env:PGDATABASE="redefined_test"
npm install
npm run migrate:up


This applies every migration in `backend/migrations/` in order, tracked in a `pgmigrations` table so re-running is always safe (already-applied migrations are skipped).

To add a new migration:

npm run migrate:create -- descriptive-name

This generates a timestamped file in `backend/migrations/` with `exports.up`/`exports.down` stubs — fill in `pgm.sql(...)` for both directions.
### 3. Configure environment variables

**PowerShell:**
```powershell
$env:PGHOST = "localhost"
$env:PGPORT = "55432"
$env:PGUSER = "redefined_test"
$env:PGPASSWORD = "redefined_test"
$env:PGDATABASE = "redefined_test"
$env:PORT = "3000"
$env:DEMO_MODE = "true"
$env:UPLOADS_DIR = "$env:TEMP\redefined-uploads"
New-Item -ItemType Directory -Force -Path $env:UPLOADS_DIR | Out-Null
```

**bash:**
```bash
export PGHOST=localhost
export PGPORT=55432
export PGUSER=redefined_test
export PGPASSWORD=redefined_test
export PGDATABASE=redefined_test
export PORT=3000
export DEMO_MODE=true
export UPLOADS_DIR=/tmp/redefined-uploads
mkdir -p /tmp/redefined-uploads
```

`DEMO_MODE=true` enables a "Buy Now (Demo)" button on the storefront that completes a purchase without needing real PayPal credentials — useful for local development and for the Playwright tests below. To exercise real PayPal checkout locally, also set `PAYPAL_CLIENT_ID`, `PAYPAL_CLIENT_SECRET`, and `PAYPAL_ENV=sandbox`.

#### The server checks its configuration before it starts

`server.ts` validates the environment at boot, reports every problem at once, and exits rather than starting — the same reasoning as the container refusing to start on a failed migration. A missing variable used to be `undefined` until the first line of code that happened to need it, which could be long after the container reported healthy, and several of those failures were silent and customer-visible.

| | Variables |
| --- | --- |
| Always required | `DEMO_MODE`, `PGHOST`, `PGPORT`, `PGUSER`, `PGPASSWORD`, `PGDATABASE`, `UPLOADS_DIR` |
| Required when `DEMO_MODE=false` | `PAYPAL_CLIENT_ID`, `PAYPAL_CLIENT_SECRET`, `PAYPAL_WEBHOOK_ID`, `PAYPAL_ENV` |
| Required when SMTP is configured | `PUBLIC_URL`, and `SMTP_USER`/`SMTP_PASSWORD` together |
| Warned about, but not fatal | SMTP absent, `ADMIN_GATE_SECRET` absent, `MAIL_ALLOWLIST` absent while SMTP is configured |

**`DEMO_MODE` must be exactly `true` or `false`.** It used to mean "demo unless the value is exactly `false`", so `DEMO_MODE=False`, `0`, or any typo left demo mode on — which meant the shop quietly stopped charging anyone. It is now required and strict, so a slip is a startup failure instead.

`PUBLIC_URL` is required only alongside SMTP because its only job is building links in email; an environment that cannot send mail does not need it. `UPLOADS_DIR` has no such reprieve — its fallback of `/app/uploads` is correct inside the container and wrong everywhere else.

**Adding to the required list has reach beyond this repository.** A variable added to `ALWAYS_REQUIRED` must also be set in every environment that deploys, and there are two of those. Both are checked automatically — `backend/tests/unit/composeEnvironment.test.ts` reads the validator's own list and fails if a compose file does not set something on it, which is what #107 existed to prevent from recurring. It runs over every deployment file in the repository and hands each one's entries to `validateEnv` itself, so a file is checked against exactly what the container checks at boot.

Production was outside that net until #118. It ran from a stack that existed only in Portainer's web editor, which no test could read — and on 2026-08-23 it refused to boot because `UPLOADS_DIR` had no line in it, while being set in Portainer's stack variables. `docker-compose.prod.yml` is now in the repository and covered like QA's, which is also why it must be deployed as a **git repository stack** rather than pasted into the web editor: a pasted copy drifts from the checked one and the guard goes back to being decorative.

Note also that setting a variable in Portainer's stack environment is not the same as giving it to the container. Stack variables are interpolated into the compose file as `${VAR}`; a service receives exactly what its own `environment:` block lists. A variable with no line there never arrives, however carefully it was set in the UI.

**Note (PowerShell):** environment variables set with `$env:` only last for the current terminal session/tab. If you close and reopen VS Code's terminal, you'll need to re-run step 3 before starting the backend again.

### 4. Run the backend

```powershell
npm install
npm run dev
```

Runs on `http://localhost:3000` with hot reload.

### 5. Run the frontend

Open a **second** terminal tab/window (the backend needs to keep running in the first one):

```powershell
cd frontend
npm install
npm run dev
```

Runs on `http://localhost:5173` and proxies `/api`, `/uploads`, `/webhooks` to the backend on port 3000.

### 6. Open it

- Storefront: http://localhost:5173
- Admin: http://localhost:5173/admin

Note: locally, `/admin` is directly reachable with no login gate — the authentik SSO protection only exists in the deployed environment (Nginx Proxy Manager + authentik forward-auth), not in local dev.

## Running tests

### Backend unit tests

No database required.

```powershell
cd backend
npm run test:unit
```

### Backend integration tests

Exercises the real Express app against a real (disposable) Postgres instance via `supertest`.

```powershell
cd backend
npm run db:test:up
npm run test:integration
npm run db:test:down   # when finished
```

### Frontend Playwright e2e tests

Needs the backend running against a database with the schema loaded (steps 1–4 above, or `.\scripts\start-local.ps1`), since these tests drive real registration/login/purchase flows through a live API.

```powershell
cd frontend
npm install
npx playwright install --with-deps   # first time only — installs browser binaries
npm run test:e2e
```

## CI

Gitea Actions runs two workflows on every push to `main` and on pull requests:

- **SonarQube Analysis** (`.gitea/workflows/sonarqube.yml`) — static analysis scan, plus TypeScript build checks for both `backend` and `frontend`
- **Tests** (`.gitea/workflows/tests.yml`) — backend unit tests, backend integration tests (against a disposable Postgres service container), and frontend Playwright e2e tests, each posting a results summary to the job's Summary tab in Gitea Actions

## Branching and commits

- All changes go on a branch off `main`, named `feature/<short-description>` or `fix/<short-description>` — never commit directly to `main`
- Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/) syntax: `feat:`, `fix:`, `chore:`, `docs:`, `test:`, `ci:`, `refactor:`, etc.
- Branches are automatically deleted after a pull request is merged

## Shipping a change

Every change follows the same path. **QA is a required stop, not an optional one** — production is not the place to discover that a deploy is wrong.

| # | Step | Gate before moving on |
| --- | --- | --- |
| 1 | Implement on a branch, verify locally | Unit, integration, and e2e suites pass; `tsc --noEmit` and `npm run build` clean |
| 2 | Commit and push | `git branch -a --contains <sha>` lists the pushed branch — a commit made after a push is silently left behind otherwise |
| 3 | Open a PR and merge to `main` | The merge commit contains the expected commits |
| 4 | **Deploy to QA and review it** | The change behaves as intended in a browser, behind authentik |
| 5 | Promote the reviewed image to production | Migrations recorded, data intact, deployed bundle is the new one |
| 6 | Stop the QA stack | — |

Steps 4 and 5 are detailed under [QA environment](#qa-environment) and [Production deployment](#production-deployment). Step 5 promotes the *same image* that was reviewed in QA, so what ships is what was tested.

## Production deployment

Production runs as a single Docker image (multi-stage build — the frontend is built to static files and served directly by the backend), deployed via Portainer behind Nginx Proxy Manager, with authentik forward-auth gating `/admin`. That infrastructure is homelab-specific and documented separately outside this repo.

The container applies pending migrations before starting the server, so deployed code can never be ahead of the database schema. A failed migration stops the container rather than letting it serve against a schema it doesn't match — check `docker logs` on the app container if it doesn't come up.

### The admin authorization boundary

Worth reading before adding any admin route, because the control is invisible from the code.

Authorization for the admin panel and the admin API lives in a single `auth_request` regex in the Nginx Proxy Manager config — `^/(admin|api/admin)` in production, and `location /` in QA, where the whole site is gated. That config is not in this repository. Three consequences follow, and none of them are visible from Express:

- **An admin route added at a path the regex does not match is not covered by it.** `/api/reports` or `/api/internal/...` would be publicly reachable the moment it shipped.
- **Anything that reaches the container directly bypasses authentik entirely**, because the gate is in the proxy in front of it. QA publishes port 32751 on the NAS and production publishes its own.
- **Locally there is no gate at all**, so `/admin` and the whole admin API are open by design and no developer ever sees the boundary being enforced.

`ADMIN_GATE_SECRET` is the application-layer half of this, and it is optional:

| State | Behaviour |
| --- | --- |
| Unset | Every admin route is reachable, exactly as before. The server logs an `[admin-gate]` warning at boot saying so, so the state is visible rather than silent. This is what local development and the test suite run in. |
| Set | Every admin router requires an `X-Admin-Gate` header matching the value, and returns 403 without it. |

To turn it on, the secret has to be set in **two places at once** — the stack's environment, and a `proxy_set_header X-Admin-Gate "<secret>";` line on the gated location in Nginx Proxy Manager. Setting it in only one of them makes the admin panel return 403 until the other catches up. That failure is loud and recoverable, unlike the one it replaces.

The middleware is attached to each admin **router** rather than to a path prefix. That is deliberate: an admin router added later at some other path inherits the gate, and because the proxy only injects the header on paths its regex matches, that router refuses on its first request rather than being quietly public. A 403 in that situation means the proxy regex needs widening — it is the boundary telling you it has drifted.

### Promoting a reviewed change to production

Only after the change has been reviewed in QA.

```bash
# 1. Back up first — the container migrates the schema on its own.
sudo docker exec -t redefined-designs-db-syn pg_dump -U redefined -d redefined \
  > /volume1/configs/redefined-designs/backup-$(date +%Y%m%d-%H%M).sql
ls -lh /volume1/configs/redefined-designs/backup-*.sql   # size must be plausible

# 2. Promote the exact image QA reviewed, rather than rebuilding a second time.
sudo docker tag redefined-designs:qa redefined-designs:latest

# 3. Verify the image before touching the running container.
sudo docker inspect --format '{{.Config.Cmd}}' redefined-designs:latest
# must print: [sh -c node migrate.js up && node dist/server.js]

# 4. Recreate the container — restarting it would keep the old image.
sudo docker stop redefined-designs-syn
sudo docker rm redefined-designs-syn
# redeploy the stack in Portainer

# 5. Verify, in this order.
sudo docker logs redefined-designs-syn | head -30
#    migration output appears BEFORE "listening on 3000",
#    and "listening on 3000" appears exactly once (repeats = crash loop)

sudo docker exec -it redefined-designs-db-syn psql -U redefined -d redefined \
  -c "SELECT name, run_on FROM pgmigrations ORDER BY run_on;"
sudo docker exec -it redefined-designs-db-syn psql -U redefined -d redefined \
  -c "SELECT count(*) FROM items;"

# 6. Confirm the deployed artifact, not just the source.
sudo docker exec redefined-designs-syn sh -c "grep -l 'must have all' /app/public/assets/*.js"
```

Then load the storefront in a private window — aggressive bundle caching has produced false "still broken" reports on this project even after a correct deploy.

**Rollback:** migrations here have been additive, so the previous image still runs against the migrated schema — retag the old image and redeploy. Restoring the SQL dump is a last resort, not a first move.

## QA environment

`docker-compose.qa.yml` defines a disposable QA stack for reviewing merged-but-undeployed changes online. It runs alongside production on the same NAS with its own containers, database, volumes, and host port, and is started only when a review is needed.

It differs from production deliberately: `DEMO_MODE=true` with no PayPal credentials (so checkout is exercisable but cannot reach live PayPal), no SMTP configuration (so it cannot email anyone), and `restart: "no"` (so a NAS reboot doesn't quietly bring it back up).

### One-time setup on the NAS

**Build the image before creating the stack.** `redefined-designs:qa` exists only on the NAS — it is never pushed to a registry. If the stack is deployed first, Compose finds no local image, falls back to pulling from Docker Hub, and fails with a misleading `pull access denied ... repository does not exist or may require 'docker login'`.

```bash
cd /volume1/docker/redefined-designs
sudo docker build --no-cache -t redefined-designs:qa /volume1/docker/redefined-designs
sudo docker images | grep redefined-designs   # confirm the qa tag is present
```

For the same reason, leave Portainer's **"Pull latest image"** toggle **off** when deploying or redeploying this stack. The compose file sets `pull_policy: never` so a missing image reports itself as missing rather than as a registry authentication problem.

Next, create the directory structure. These paths are deliberately **not** under `/volume1/configs/redefined-designs` — sharing production's Postgres directory would mean QA writing into production's database files.

```bash
sudo mkdir -p /volume1/configs/redefined-designs-qa/postgres
sudo mkdir -p /volume1/configs/redefined-designs-qa/uploads

# The official postgres image runs as uid/gid 999. Without this the DB
# container exits immediately with a data-directory permissions error.
sudo chown -R 999:999 /volume1/configs/redefined-designs-qa/postgres
sudo chmod 700 /volume1/configs/redefined-designs-qa/postgres

# Uploads are written by the app container, which runs as root.
sudo chown -R root:root /volume1/configs/redefined-designs-qa/uploads
sudo chmod 755 /volume1/configs/redefined-designs-qa/uploads

# Confirm the two environments are separate before going further.
ls -la /volume1/configs/redefined-designs-qa/
ls -la /volume1/configs/redefined-designs/
```

Then, in Portainer, create a stack **named `redefined-designs-qa`** from `docker-compose.qa.yml`, with one environment variable `QA_DB_PASSWORD`. The stack name matters: it becomes the compose project name, and reusing production's name would make compose reconcile the two stacks against each other and remove the production containers.

Finally, add an Nginx Proxy Manager host for `qa-redefined-designs` pointing at the NAS on port `32751`, with authentik forward-auth on `location /` — the whole site, not just `/admin`, so nothing unreleased is publicly reachable.

### Reviewing a change

```bash
cd /volume1/docker/redefined-designs
gitc() { sudo docker run --rm -it -v /volume1/docker/redefined-designs:/repo -v ~/.gitconfig-docker/.gitconfig:/root/.gitconfig -w /repo alpine/git "$@"; }
gitc pull origin main
gitc log --oneline -3

# Rebuild the QA image before restarting the stack — the stack does not build
# anything itself, and will otherwise run whatever was last tagged :qa.
sudo docker build --no-cache -t redefined-designs:qa /volume1/docker/redefined-designs
# start the redefined-designs-qa stack in Portainer

# Migrations run as the container starts — confirm before reviewing.
sudo docker logs redefined-designs-qa-syn | head -30
```

Stop the stack in Portainer when finished.

### Resetting QA data

QA data is disposable. To start from an empty database, stop the stack first, then:

```bash
# Check the path before running this. It must contain -qa.
sudo rm -rf /volume1/configs/redefined-designs-qa/postgres/pgdata
```

Restarting the stack recreates the database and re-runs every migration from scratch.