Commit Graph
5 Commits
Author SHA1 Message Date
bermudalamb 167c8ad97c fix(uploads): ship the image backfill script in the container image (#231)
Linting / lint (pull_request) Successful in 3m48s
SonarQube Analysis / sonarqube (pull_request) Failing after 21m6s
`npm run backfill:images` could not run in QA or production. Three reasons, each sufficient alone: tsconfig includes only `src`, so `scripts/` was never compiled; the Dockerfile copies `dist`, `migrate.js` and `migrations` and never `scripts/`; and `tsx`, which the npm script invoked, is a devDependency that `npm install --omit=dev` strips from the final stage. The half of #226 that closes the exposure on already-stored photos had no way to run where the photos are.

Moved to `src/backfillImageReencode.ts` so it compiles into `dist` and ships. Both of its runtime dependencies, sharp and pg, were already production dependencies, so the image needs nothing else. `scripts/bench-hash-latency.ts` was the pattern followed originally, and it is a development tool that never needs to run deployed; this one is an operational task that can only be useful where the images are, which makes `migrate.js` the right precedent instead.

The npm script now runs the compiled output rather than tsx, so one command behaves identically on a laptop and inside a container. The entry point is guarded with `require.main === module`: putting a catalogue-wide irreversible rewrite in the same directory the server imports at boot means an accidental import would otherwise run it, and nothing should depend on people continuing not to write that import.

Proven in the built production image rather than argued. `node_modules/.bin/tsx` and `scripts/` are both absent from it, and `npm run backfill:images` still runs: report mode found the planted file, `--apply` rewrote it 35760 to 16019 bytes, a second `--apply` reported skipped 1 processed 0, and on the mounted volume the EXIF was gone with the image bounded to 2000x1333 and still JPEG. That is the exact scenario the previous version would have failed.

Backend: 285 unit, 260 integration, tsc clean, lint unchanged at six warnings, all six pre-existing.

Closes #231
2026-08-29 15:03:44 -05:00
bermudalamb 502d56d9fd feat(uploads): backfill re-encoding over already-stored photos (#226)
Linting / lint (pull_request) Successful in 1m56s
SonarQube Analysis / sonarqube (pull_request) Failing after 16m51s
Stripping new uploads does nothing for the catalogue that is already on the storefront, which is where the exposure actually lives today. This is the other half.

Reports by default and rewrites nothing without --apply, because the transform is lossy and there is no undo. Idempotency comes from `needsProcessing` rather than from a marker or a schema change: a file with no EXIF already inside the bounds is already in its final state, so a second run skips it instead of putting it through another lossy pass. Proven rather than assumed — a second --apply immediately after the first reports skipped 1, processed 0.

Verified end to end against a real row and a real file. 3000x2000 carrying GPS EXIF became 2000x1333 with the metadata gone, 35760 bytes down to 16019, the format preserved, no temporary file left behind, and `item_images.image_path` untouched. That last part is what preserving the format bought: the backfill rewrites bytes and writes nothing to the database, so there is no window where a row points at a file that no longer exists.

Both degenerate branches are exercised too, since a script that dies partway through a catalogue leaves the rest of it exposed: a row pointing at a missing file and a row with an extension the application would refuse to serve are each reported and counted, and the run continues.

`handleRow` is split out of `run` for cognitive complexity, and while doing that a miscount was introduced and caught — incrementing `processed` before the rewrite meant a file that threw would have been counted as both processed and failed, which makes the summary unreadable at the moment it matters most.

Ref #226
2026-08-29 13:51:26 -05:00
bermudalamb 313b48f582 test(perf): measure what concurrent hashing actually costs a bystander request (#163)
Linting / lint (pull_request) Successful in 2m5s
SonarQube Analysis / sonarqube (pull_request) Failing after 5m9s
#163 claimed `bcryptjs` blocks the event loop and that every login stalls every other request in flight. That was asserted without measurement and is wrong: the asynchronous API chunks its work and yields between rounds, and all six call sites in `routes/customers.ts` use it — there is no `hashSync` or `compareSync` anywhere in `src`.

A smaller effect is real, though, and this measures it instead of arguing about it. The probe is `GET /api/customers/me` with no cookie, chosen for doing almost nothing: it rejects before touching the database, so nearly all of its latency is time spent waiting for the event loop rather than work of its own. The load is real registrations against the real route, because the question is what a deployed server does rather than what bcrypt does on a bench.

Measured on the dev machine, Node 24, cost 12:

| Concurrent registrations | Each registration (p50) | Bystander p95 | Bystander worst case |
| --- | --- | --- | --- |
| idle | — | 0.3 ms | 5.4 ms |
| 1 | 213 ms | 0.6 ms | 102 ms |
| 4 | 803 ms | 101.9 ms | 405 ms |
| 8 | 1626 ms | 15.6 ms | 808 ms |

Both columns are linear in the number of queued hashes. Registration is roughly 200 ms times the concurrency, because the hashes serialize onto the one thread. The bystander's worst case is roughly 100 ms times the concurrency, which matches the coarseness of the chunks measured earlier — about 100 ms of un-yielding time per hash.

The median stays under a millisecond throughout, so this is a tail-latency characteristic and not the stall the issue described.

The benchmark creates real customers and deletes them again, because it is normally pointed at a development database that nothing truncates.

Lint now covers `scripts` as well as `src`, so the one file in it is held to the same standard as the rest.
2026-08-24 16:59:33 -05:00
bermudalamb 4b524e0da3 fix: move scripts folder to correct location
SonarQube Analysis / sonarqube (pull_request) Successful in 4m31s
Tests / backend-unit (pull_request) Successful in 59s
Tests / backend-integration (pull_request) Failing after 4s
Tests / frontend-e2e (pull_request) Failing after 1s
2026-08-14 08:15:52 -05:00
bermudalamb 234330e482 Add CI test workflow with job summaries
SonarQube Analysis / sonarqube (push) Successful in 3m55s
Tests / backend-unit (push) Failing after 55s
Tests / backend-integration (push) Failing after 2s
Tests / frontend-e2e (push) Failing after 1s
2026-08-13 19:10:30 -05:00