redefined-designs/docs/superpowers/specs/2026-08-29-intake-pipeline-design.md

Intake Pipeline — Design

Issue: #220 — Shared upload links, AI-drafted listings, and an admin review queue Date: 2026-08-29 Status: Draft

Goal

Someone who is not the admin can send in photos of one item through a link they were given. A draft listing is written for it automatically. The admin is told by email, and the item sits unpublished until the admin has read the copy, typed a price and published it deliberately.

Where this starts from

Most of the lifecycle this feature needs is already built, which is why the design below adds a pipeline in front of it rather than a parallel one beside it.

• pending is already the unpublished state. items.status defaults to 'pending' since #90, NON_PUBLIC_STATUSES keeps it out of every public and storefront query, and POST /admin/items/:id/mark-available and /unpublish already move an item across that line. Nothing here needs a new visibility concept, and inventing one would mean a second thing to keep correct.
• Uploads are already hardened. Multer writes to UPLOADS_DIR, uploadTypes.ts checks the declared type against the file's leading bytes, the allowlist is three image types, and uploads.ts refuses to serve anything it does not recognise. A new upload path that did not go through those is a new hole.
• Mail already exists, including admin-editable markdown bodies with placeholder validation, the MAIL_ALLOWLIST guard that stops non-production environments mailing real people, and PUBLIC_URL for building links.
• Background work has a pattern. cron.schedule('0 9 * * *', ...) in server.ts drives the cart reminders. This is a single-instance deployment, which is what makes in-process scheduling and the in-memory rate-limit store tenable.

Three things do not exist: a way in for someone without an admin account, any LLM integration at all, and anywhere to review a draft.

Decisions

Settled in conversation before this was written, and recorded on the issue.

Question	Decision
What the "shared folder" is	A web upload page, not a watched NAS share or a mailbox
Who can upload	Anyone holding a named, revocable link — no account
What the uploader supplies	Photos of one item, plus one free-text note
What the AI drafts	Name, marketing description, category and tags from the existing taxonomy, and a suggested price
Where a submission lives pre-approval	An items row at status='pending', not a separate submissions table
How the admin acts on it	Signed one-click links in the email, plus a review queue in the admin
Whether one click can publish	No. Publishing always requires a typed price
Model	Sonnet 5, in an env var

The invariant

Nothing reaches the storefront at a price a model guessed. Everything below follows from that: the suggested price lives on the draft and never on the item, the email has no publish button, and the publish path refuses an item with no price.

Architecture

Schema

Three migrations.

CREATE TABLE upload_links (
  id SERIAL PRIMARY KEY,
  label TEXT NOT NULL,
  -- The token is never stored. A leaked database is not also a leaked set of
  -- working upload links, and the admin screen can show a token exactly once,
  -- at creation, for the same reason a password reset link is not re-readable.
  token_hash TEXT NOT NULL UNIQUE,
  revoked_at TIMESTAMPTZ,
  submission_count INTEGER NOT NULL DEFAULT 0,
  -- Null means no cap. A link handed to a regular contributor is open-ended;
  -- one handed out for a single box of stock is not.
  max_submissions INTEGER,
  last_used_at TIMESTAMPTZ,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE TABLE item_drafts (
  id SERIAL PRIMARY KEY,
  item_id INTEGER NOT NULL UNIQUE REFERENCES items(id) ON DELETE CASCADE,
  -- SET NULL rather than CASCADE: deleting a link must not delete the items
  -- that came in through it. Provenance is lost; the goods are not.
  upload_link_id INTEGER REFERENCES upload_links(id) ON DELETE SET NULL,
  submitter_note TEXT,
  state TEXT NOT NULL DEFAULT 'queued',   -- queued | drafting | ready | failed | discarded
  attempts INTEGER NOT NULL DEFAULT 0,
  model TEXT,
  ai_name TEXT,
  ai_description TEXT,
  ai_category_id INTEGER REFERENCES categories(id) ON DELETE SET NULL,
  ai_tag_names TEXT[],
  -- Deliberately not on items. See the invariant.
  ai_suggested_price_cents INTEGER,
  ai_error TEXT,
  input_tokens INTEGER,
  output_tokens INTEGER,
  cost_micros INTEGER,
  drafted_at TIMESTAMPTZ,
  reviewed_at TIMESTAMPTZ,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

ALTER TABLE items ALTER COLUMN price_cents DROP NOT NULL;

On making price_cents nullable. This is the most invasive line in the design: price_cents is referenced across fifteen files including the cart and the checkout. The alternative is to insert a submission at 0 and guard publishing on price_cents > 0, which changes no types and no other file.

It is still the wrong choice. Zero is a real number that formats as a real price, and a sentinel that renders as "$0.00" in a shop is precisely the failure this codebase already has a scar from — DEMO_MODE meaning "demo unless the value is exactly false" quietly stopped the shop charging anyone. Null cannot be formatted by accident. And the blast radius is not hidden work but the opposite: making the column nullable makes the TypeScript type number | null, and the compiler then names every one of those fifteen sites and refuses to build until each has said what it does with an unpriced item. A sentinel gets the same reach with none of the enumeration.

The publish guard belongs in mark-available alongside the existing status checks, and is worth an integration test of its own.

Ingestion

POST /api/intake/:token — public, unauthenticated, uploadImages for the files.

1. Hash the token, look up a matching upload_links row that is not revoked. No match, revoked, or over its cap: 404. Not 403 — whether a link exists is not something a stranger needs to be able to distinguish, which is the reasoning uploads.ts already applies.
2. Validate the photos through the existing uploadTypes checks. Caps: at most 10 photos, at most 10 MB each.
3. In one transaction: insert the items row (status='pending', price_cents=NULL, a placeholder name — the submission timestamp — replaced by the draft or the admin), insert item_images, insert item_drafts at state='queued', bump the link's counter.
4. Respond immediately. The uploader is told it arrived and that a person will look at it. The AI is not called here — a slow or failing API call must not turn into a failed upload for someone who did nothing wrong.

Rate limited by IP through a new limiter in rateLimit.ts. keyByCallerAndEmail does not fit — there is no email — so this needs its own key function over ipKeyGenerator alone, and the existing comment about a bare ip: bucket being a shared allowance applies and is accepted here: the link itself is the per-caller identity, and its counter is the per-caller cap.

The drafting worker

A new backend/src/intakeDrafting.ts, driven both by a call at the end of a successful submission and by a node-cron sweeper that picks up anything left queued or stuck in drafting. The sweeper is what makes a restart mid-draft recoverable rather than a permanently stalled row.

The request, via @anthropic-ai/sdk and client.messages.parse() with zodOutputFormat, so the shape is validated rather than parsed out of prose:

• Every photo as a base64 image block.
• The submitter's note verbatim, clearly framed as the only trustworthy factual claims available. The system prompt says plainly: describe what is visible, use the note for anything not visible, and never state a material, age, maker or provenance that is in neither. On a one-of-a-kind item, an invented "1930s hand-thrown stoneware" is not a cosmetic error — it is a false claim on a storefront.
• The existing categories and tags as the closed set to choose from, so the draft lands inside the taxonomy the filters already work on.
• The suggested price framed as a rough starting point.

The note is untrusted input from an unauthenticated stranger. It is passed as data, and nothing the model returns is executed, interpolated into SQL, or rendered as HTML — the drafted description goes through the same markdown-with-html: false treatment as every other stored body, which is what makes "a model wrote this" and "a person wrote this" equally safe to render.

Result: state='ready', the fields populated, token counts and computed cost recorded. Failure: attempts incremented, ai_error stored, and after three attempts state='failed' — which still leaves a perfectly good pending item with photos in the review queue, just with no draft copy. A failed AI call must never lose someone's submission.

Spend ceiling. A monthly cap (INTAKE_MONTHLY_BUDGET_USD) summed from cost_micros. Past it, submissions still save and still notify; only the API call is skipped, with state='failed' and an error saying so. The ceiling protects the bill, and it must not be the thing that loses inventory.

Notification and signed links

Reuses emailTemplates.ts with a new intakeDraft key, so the copy is editable from the settings screen like every other template, with reviewUrl required.

Action links are HMAC-signed with a new INTAKE_ACTION_SECRET over (draftId, action, expiry), verified with a timing-safe comparison, good for 30 days.

• Regenerate — back to state='queued'.
• Discard — state='discarded' and the item unpublished; recoverable from the queue, because a one-click destructive action reachable from an inbox should not be final.
• Review & publish — an ordinary deep link into the admin queue, behind authentik like the rest of /admin. It carries no signature and grants nothing.

The two signed actions are deliberately the ones whose worst case is a wasted API call or a recoverable hide. Nothing a single click can do puts an item on the storefront.

The review queue

A new admin screen listing drafts by state: photos, the submitter's note, which link it came from, the drafted copy in editable fields, the suggested price shown as a suggestion beside an empty price input, and Publish / Regenerate / Discard. Publish writes the edited values onto the item and calls the existing mark-available.

Reuses the existing admin item components where they fit rather than growing a second item editor.

Configuration

New variables, added to envValidation.ts and — per #107, enforced by composeEnvironment.test.ts — to both compose files if they go on the required list.

Variable	Requirement
ANTHROPIC_API_KEY	Required when intake is enabled
INTAKE_MODEL	Optional, defaults to claude-sonnet-5
INTAKE_MONTHLY_BUDGET_USD	Optional, defaults to 20
INTAKE_ACTION_SECRET	Required when intake is enabled

Intake is feature-flagged off by default, so none of these become required for an environment that does not run it. PUBLIC_URL is already required alongside SMTP and is what the review links are built from.

Error handling

The through-line: a submission is the only irreplaceable thing here. Photos of a one-of-a-kind object may not be retakeable — the item may not be in the sender's hands any more. Every failure mode below is arranged so the photos survive it.

Failure	Behaviour
API call fails or times out	Retry to three attempts, then state='failed'. Item and photos intact, reviewable with no draft
Budget exhausted	Submission saved, draft skipped, admin still notified
Malformed model output	Rejected by the schema, counts as a failed attempt
SMTP down	Draft still ready and visible in the queue; the queue, not the email, is the source of truth
Link revoked mid-upload	404. Already-submitted items are unaffected
Restart mid-draft	The cron sweeper re-queues anything stuck in drafting

Testing

• Unit — token hashing and constant-time verification, HMAC signing and expiry, the budget calculation, the prompt builder's handling of an empty note, and the publish guard's price rule. All pure, in the style of keyByCallerAndEmail and isAllowedRecipient being exported specifically to be tested directly.
• Integration — submission through a valid link creates a pending item with images and a queued draft; a revoked link gets 404; publishing without a price is refused; a signed action link works once and an expired or tampered one does not. The Anthropic client is stubbed; no test spends money.
• E2E — the submission page, and the queue round trip from draft to published. Assertions scoped to the item under test rather than the whole grid, since the dev database never truncates.

Out of scope

Contributor accounts, a watched NAS folder, email-in submission, multi-item submissions, image editing or cropping, and regenerating with a steer ("try again, warmer"). All are additive later; none change the schema above.