Unblocks the backfill half of #226. New uploads were already verified working in QA; this is what lets the already-stored photos be fixed too.
The problem
npm run backfill:images could not run in QA or production. Three reasons, each sufficient on its own:
tsconfig.json declares "include": ["src"], so scripts/ was never compiled — no dist/scripts
The Dockerfile copies dist, migrate.js and migrations; scripts/ never enters the image
tsx, which the npm script invoked, is a devDependency stripped by npm install --omit=dev
The script existed only in the repository, never where the photos are.
The fix
Moved to src/backfillImageReencode.ts so it compiles into dist and ships. Both runtime dependencies — sharp and pg — were already production dependencies, so the image needs nothing else.
scripts/bench-hash-latency.ts was the pattern followed originally, and it is a development tool that never needs to run deployed. This script has the opposite requirement, which makes migrate.js the right precedent: an operational entry point at a path the image carries.
The npm script now runs the compiled output rather than tsx, so one command behaves identically on a laptop and inside a container. The entry point is guarded with require.main === module — putting a catalogue-wide, irreversible rewrite in the directory the server imports at boot means an accidental import would otherwise execute it, and that should not depend on nobody ever writing that import.
Proven in the built image, not argued
$ docker run --rm <image> sh -c "ls node_modules/.bin/tsx; ls scripts"
ls: cannot access 'node_modules/.bin/tsx': No such file or directory
ls: cannot access 'scripts': No such file or directory
$ docker run --rm ... <image> npm run backfill:images
[backfill] would process /uploads/old-photo.jpg (3000x2000, exif present, 35760 bytes)
$ docker run --rm ... <image> npm run backfill:images -- --apply
[backfill] /uploads/old-photo.jpg: 35760 -> 16019 bytes processed: 1
$ docker run --rm ... <image> npm run backfill:images -- --apply
skipped: 1, processed: 0
On the mounted volume afterwards: { exif: 'GONE', w: 2000, h: 1333, format: 'jpeg' }, and no leftover files. That is precisely the scenario the previous version would have failed — the image has no tsx and no scripts/, and it still works.
Backend: 285 unit, 260 integration, tsc clean, lint unchanged at six warnings (all pre-existing — three from the Drizzle spike, three older).
How this was missed
I wrote and tested the script locally and never checked that it ships. It was caught while writing the QA verification checklist rather than by you finding it on the NAS, which is the only good thing about it.
After merge
Task 5 Phase 3 becomes runnable:
docker exec redefined-designs-qa-syn npm run backfill:images # report
docker exec redefined-designs-qa-syn npm run backfill:images -- --apply
docker exec redefined-designs-qa-syn npm run backfill:images -- --apply # must skip
QA first, look at the images by eye, then back up production's uploads volume and confirm the restore before applying there.
Unblocks the backfill half of #226. New uploads were already verified working in QA; this is what lets the **already-stored** photos be fixed too.
## The problem
`npm run backfill:images` could not run in QA or production. Three reasons, each sufficient on its own:
- `tsconfig.json` declares `"include": ["src"]`, so `scripts/` was never compiled — no `dist/scripts`
- The Dockerfile copies `dist`, `migrate.js` and `migrations`; `scripts/` never enters the image
- `tsx`, which the npm script invoked, is a devDependency stripped by `npm install --omit=dev`
The script existed only in the repository, never where the photos are.
## The fix
Moved to `src/backfillImageReencode.ts` so it compiles into `dist` and ships. Both runtime dependencies — `sharp` and `pg` — were already production dependencies, so the image needs nothing else.
`scripts/bench-hash-latency.ts` was the pattern followed originally, and it is a development tool that never needs to run deployed. This script has the opposite requirement, which makes `migrate.js` the right precedent: an operational entry point at a path the image carries.
The npm script now runs the compiled output rather than `tsx`, so one command behaves identically on a laptop and inside a container. The entry point is guarded with `require.main === module` — putting a catalogue-wide, irreversible rewrite in the directory the server imports at boot means an accidental import would otherwise execute it, and that should not depend on nobody ever writing that import.
## Proven in the built image, not argued
```
$ docker run --rm <image> sh -c "ls node_modules/.bin/tsx; ls scripts"
ls: cannot access 'node_modules/.bin/tsx': No such file or directory
ls: cannot access 'scripts': No such file or directory
$ docker run --rm ... <image> npm run backfill:images
[backfill] would process /uploads/old-photo.jpg (3000x2000, exif present, 35760 bytes)
$ docker run --rm ... <image> npm run backfill:images -- --apply
[backfill] /uploads/old-photo.jpg: 35760 -> 16019 bytes processed: 1
$ docker run --rm ... <image> npm run backfill:images -- --apply
skipped: 1, processed: 0
```
On the mounted volume afterwards: `{ exif: 'GONE', w: 2000, h: 1333, format: 'jpeg' }`, and no leftover files. That is precisely the scenario the previous version would have failed — the image has no `tsx` and no `scripts/`, and it still works.
Backend: **285 unit**, **260 integration**, tsc clean, lint unchanged at six warnings (all pre-existing — three from the Drizzle spike, three older).
## How this was missed
I wrote and tested the script locally and never checked that it ships. It was caught while writing the QA verification checklist rather than by you finding it on the NAS, which is the only good thing about it.
## After merge
Task 5 Phase 3 becomes runnable:
```
docker exec redefined-designs-qa-syn npm run backfill:images # report
docker exec redefined-designs-qa-syn npm run backfill:images -- --apply
docker exec redefined-designs-qa-syn npm run backfill:images -- --apply # must skip
```
QA first, look at the images by eye, then back up production's uploads volume and confirm the restore before applying there.
Closes #231
`npm run backfill:images` could not run in QA or production. Three reasons, each sufficient alone: tsconfig includes only `src`, so `scripts/` was never compiled; the Dockerfile copies `dist`, `migrate.js` and `migrations` and never `scripts/`; and `tsx`, which the npm script invoked, is a devDependency that `npm install --omit=dev` strips from the final stage. The half of #226 that closes the exposure on already-stored photos had no way to run where the photos are.
Moved to `src/backfillImageReencode.ts` so it compiles into `dist` and ships. Both of its runtime dependencies, sharp and pg, were already production dependencies, so the image needs nothing else. `scripts/bench-hash-latency.ts` was the pattern followed originally, and it is a development tool that never needs to run deployed; this one is an operational task that can only be useful where the images are, which makes `migrate.js` the right precedent instead.
The npm script now runs the compiled output rather than tsx, so one command behaves identically on a laptop and inside a container. The entry point is guarded with `require.main === module`: putting a catalogue-wide irreversible rewrite in the same directory the server imports at boot means an accidental import would otherwise run it, and nothing should depend on people continuing not to write that import.
Proven in the built production image rather than argued. `node_modules/.bin/tsx` and `scripts/` are both absent from it, and `npm run backfill:images` still runs: report mode found the planted file, `--apply` rewrote it 35760 to 16019 bytes, a second `--apply` reported skipped 1 processed 0, and on the mounted volume the EXIF was gone with the image bounded to 2000x1333 and still JPEG. That is the exact scenario the previous version would have failed.
Backend: 285 unit, 260 integration, tsc clean, lint unchanged at six warnings, all six pre-existing.
Closes#231
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Unblocks the backfill half of #226. New uploads were already verified working in QA; this is what lets the already-stored photos be fixed too.
The problem
npm run backfill:imagescould not run in QA or production. Three reasons, each sufficient on its own:tsconfig.jsondeclares"include": ["src"], soscripts/was never compiled — nodist/scriptsdist,migrate.jsandmigrations;scripts/never enters the imagetsx, which the npm script invoked, is a devDependency stripped bynpm install --omit=devThe script existed only in the repository, never where the photos are.
The fix
Moved to
src/backfillImageReencode.tsso it compiles intodistand ships. Both runtime dependencies —sharpandpg— were already production dependencies, so the image needs nothing else.scripts/bench-hash-latency.tswas the pattern followed originally, and it is a development tool that never needs to run deployed. This script has the opposite requirement, which makesmigrate.jsthe right precedent: an operational entry point at a path the image carries.The npm script now runs the compiled output rather than
tsx, so one command behaves identically on a laptop and inside a container. The entry point is guarded withrequire.main === module— putting a catalogue-wide, irreversible rewrite in the directory the server imports at boot means an accidental import would otherwise execute it, and that should not depend on nobody ever writing that import.Proven in the built image, not argued
On the mounted volume afterwards:
{ exif: 'GONE', w: 2000, h: 1333, format: 'jpeg' }, and no leftover files. That is precisely the scenario the previous version would have failed — the image has notsxand noscripts/, and it still works.Backend: 285 unit, 260 integration, tsc clean, lint unchanged at six warnings (all pre-existing — three from the Drizzle spike, three older).
How this was missed
I wrote and tested the script locally and never checked that it ships. It was caught while writing the QA verification checklist rather than by you finding it on the NAS, which is the only good thing about it.
After merge
Task 5 Phase 3 becomes runnable:
QA first, look at the images by eye, then back up production's uploads volume and confirm the restore before applying there.
Closes #231