redefined-designs/docs/superpowers/specs/2026-08-29-intake-pipeline-design.md

Intake Pipeline — Design

Issue: #220 — Shared upload links, AI-drafted listings, and an admin review queue Date: 2026-08-29 Status: Draft

Goal

Someone who is not the admin can send in photos of one item through a link they were given. A draft listing is written for it automatically. The admin is told by email, and the item sits unpublished until the admin has read the copy, checked the price and published it deliberately.

Where this starts from

Most of the lifecycle this feature needs is already built, which is why the design below adds a pipeline in front of it rather than a parallel one beside it.

• pending is already the unpublished state. items.status defaults to 'pending' since #90, NON_PUBLIC_STATUSES keeps it out of every public and storefront query, and POST /admin/items/:id/mark-available and /unpublish already move an item across that line. Nothing here needs a new visibility concept, and inventing one would mean a second thing to keep correct.
• Uploads are already hardened. Multer writes to UPLOADS_DIR, uploadTypes.ts checks the declared type against the file's leading bytes, the allowlist is three image types, and uploads.ts refuses to serve anything it does not recognise. A new upload path that did not go through those is a new hole.
• Mail already exists, including admin-editable markdown bodies with placeholder validation, the MAIL_ALLOWLIST guard that stops non-production environments mailing real people, and PUBLIC_URL for building links.
• Background work has a pattern. cron.schedule('0 9 * * *', ...) in server.ts drives the cart reminders. This is a single-instance deployment, which is what makes in-process scheduling and the in-memory rate-limit store tenable.

Three things do not exist: a way in for someone without an admin account, any LLM integration at all, and anywhere to review a draft.

Decisions

Settled in conversation before this was written, and recorded on the issue.

Question	Decision
What the "shared folder" is	A web upload page, not a watched NAS share or a mailbox
Who can upload	Anyone holding a named, revocable link — no account
What the uploader supplies	Photos of one item, plus one free-text note
What the AI drafts	Name, marketing description, category and tags from the existing taxonomy, and a suggested price
Where a submission lives pre-approval	An items row at status='pending', not a separate submissions table
How the admin acts on it	Signed one-click links in the email, plus a review queue in the admin
Whether one click can publish	No. Publishing always happens from the review queue
What an item is priced at on arrival	The AI's suggestion, or 80.00 when it has none
Model	Sonnet 5, in an env var

The invariant

Nothing reaches the storefront without the admin publishing it from the queue. The item arrives pending, which is already invisible to every public query, and only mark-available moves it. The email carries no publish button, so the admin has necessarily seen the item and its price before anything ships.

The price is the deliberate exception, and it is worth being precise about what was traded. An arriving item is priced immediately — the model's suggestion if it made one, otherwise 80.00 — so the review queue's price field is pre-filled rather than blank. The admin can change it, but is not forced to, and an unchanged field publishes at a number the admin did not choose.

That is a real risk and it is accepted knowingly. It is bounded by the queue: publishing is a deliberate act on a screen showing the price, not something that can happen from an inbox or by a timeout. What is given up is the stronger guarantee that a machine-guessed price could never reach the storefront at all.

Two consequences follow, and both are load-bearing:

• 80.00 is a plausible price, not an obvious sentinel. 0 would render as "$0.00" and read as a bug to anyone who saw it; 80.00 renders as a decision. A default that went unnoticed therefore sells the item rather than announcing itself. This is the argument for surfacing it loudly in the queue rather than for choosing a different number.
• The queue must make the price's provenance visible. A price field is not enough. The queue shows whether the number came from the model, from the 80.00 default, or from the admin, so "nobody has looked at this price" is legible at a glance instead of being indistinguishable from a considered one.

Architecture

Schema

Three migrations.

CREATE TABLE upload_links (
  id SERIAL PRIMARY KEY,
  label TEXT NOT NULL,
  -- The token is never stored. A leaked database is not also a leaked set of
  -- working upload links, and the admin screen can show a token exactly once,
  -- at creation, for the same reason a password reset link is not re-readable.
  token_hash TEXT NOT NULL UNIQUE,
  revoked_at TIMESTAMPTZ,
  submission_count INTEGER NOT NULL DEFAULT 0,
  -- Null means no cap. A link handed to a regular contributor is open-ended;
  -- one handed out for a single box of stock is not.
  max_submissions INTEGER,
  last_used_at TIMESTAMPTZ,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE TABLE item_drafts (
  id SERIAL PRIMARY KEY,
  item_id INTEGER NOT NULL UNIQUE REFERENCES items(id) ON DELETE CASCADE,
  -- SET NULL rather than CASCADE: deleting a link must not delete the items
  -- that came in through it. Provenance is lost; the goods are not.
  upload_link_id INTEGER REFERENCES upload_links(id) ON DELETE SET NULL,
  submitter_note TEXT,
  state TEXT NOT NULL DEFAULT 'queued',   -- queued | drafting | ready | failed | discarded
  attempts INTEGER NOT NULL DEFAULT 0,
  model TEXT,
  ai_name TEXT,
  ai_description TEXT,
  ai_category_id INTEGER REFERENCES categories(id) ON DELETE SET NULL,
  ai_tag_names TEXT[],
  -- Kept even though the suggestion is also copied onto the item, so what the
  -- model proposed stays readable after the admin has edited the item's price.
  -- Without it there is no way to ask later whether the model's numbers were
  -- any good.
  ai_suggested_price_cents INTEGER,
  -- ai | default | admin. What the item's current price actually came from.
  -- Derivable by comparing three numbers, but only fragilely: a model that
  -- happens to suggest exactly 8000, or an admin who deliberately types the
  -- model's number, both collapse the comparison. Recorded rather than
  -- inferred, because the queue uses it to say "nobody has chosen this price".
  price_source TEXT NOT NULL DEFAULT 'default',
  ai_error TEXT,
  input_tokens INTEGER,
  output_tokens INTEGER,
  cost_micros INTEGER,
  drafted_at TIMESTAMPTZ,
  reviewed_at TIMESTAMPTZ,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

-- An arriving item is always priced, so the column stays NOT NULL and only
-- gains a fallback. 80.00 applies when the model declined to suggest anything;
-- a suggestion, when there is one, is written over it by the worker.
ALTER TABLE items ALTER COLUMN price_cents SET DEFAULT 8000;

On price_cents staying NOT NULL. The alternative considered was making it nullable so that "no price yet" was expressible, with the publish path refusing an unpriced item. That was rejected in favour of always having a price.

The practical effect is that this design costs far less to build. price_cents is referenced across fifteen files including the cart and the checkout, and every one of them keeps working untouched — the column's type does not change, so nothing downstream has to learn about an item that has no price. The migration adds a default and nothing else.

What it costs is covered under the invariant above: the storefront can now be reached by a price the admin never chose, so the protection moves from the schema into the queue's presentation, where it is weaker. price_source exists to make that presentation possible.

The number lives in the migration rather than in configuration. Changing a default price is a rare, deliberate act with a record attached, which is the right amount of friction; an env var would let it drift silently between environments, and a wrong default is not visible anywhere until something has already sold at it.

Ingestion

POST /api/intake/:token — public, unauthenticated, uploadImages for the files.

1. Hash the token, look up a matching upload_links row that is not revoked. No match, revoked, or over its cap: 404. Not 403 — whether a link exists is not something a stranger needs to be able to distinguish, which is the reasoning uploads.ts already applies.
2. Validate the photos through the existing uploadTypes checks. Caps: at most 10 photos, at most 10 MB each.
3. In one transaction: insert the items row (status='pending', no price_cents given so the 8000 default applies, a placeholder name — the submission timestamp — replaced by the draft or the admin), insert item_images, insert item_drafts at state='queued' and price_source='default', bump the link's counter.
4. Respond immediately. The uploader is told it arrived and that a person will look at it. The AI is not called here — a slow or failing API call must not turn into a failed upload for someone who did nothing wrong.

Rate limited by IP through a new limiter in rateLimit.ts. keyByCallerAndEmail does not fit — there is no email — so this needs its own key function over ipKeyGenerator alone, and the existing comment about a bare ip: bucket being a shared allowance applies and is accepted here: the link itself is the per-caller identity, and its counter is the per-caller cap.

The drafting worker

A new backend/src/intakeDrafting.ts, driven both by a call at the end of a successful submission and by a node-cron sweeper that picks up anything left queued or stuck in drafting. The sweeper is what makes a restart mid-draft recoverable rather than a permanently stalled row.

The request, via @anthropic-ai/sdk and client.messages.parse() with zodOutputFormat, so the shape is validated rather than parsed out of prose:

• Every photo as a base64 image block.
• The submitter's note verbatim, clearly framed as the only trustworthy factual claims available. The system prompt says plainly: describe what is visible, use the note for anything not visible, and never state a material, age, maker or provenance that is in neither. On a one-of-a-kind item, an invented "1930s hand-thrown stoneware" is not a cosmetic error — it is a false claim on a storefront.
• The existing categories and tags as the closed set to choose from, so the draft lands inside the taxonomy the filters already work on.
• The suggested price framed as a rough starting point.

The note is untrusted input from an unauthenticated stranger. It is passed as data, and nothing the model returns is executed, interpolated into SQL, or rendered as HTML — the drafted description goes through the same markdown-with-html: false treatment as every other stored body, which is what makes "a model wrote this" and "a person wrote this" equally safe to render.

Result: state='ready', the fields populated, token counts and computed cost recorded. Where the model returned a price, it is written onto the item and price_source='ai'; where it did not, the item keeps the 8000 default and price_source stays 'default'. The suggested price is recorded on the draft either way.

Failure: attempts incremented, ai_error stored, and after three attempts state='failed' — which still leaves a perfectly good pending item with photos in the review queue, just with no draft copy and priced at the default. A failed AI call must never lose someone's submission.

Spend ceiling. A monthly cap (INTAKE_MONTHLY_BUDGET_USD) summed from cost_micros. Past it, submissions still save and still notify; only the API call is skipped, with state='failed' and an error saying so. The ceiling protects the bill, and it must not be the thing that loses inventory.

Notification and signed links

Reuses emailTemplates.ts with a new intakeDraft key, so the copy is editable from the settings screen like every other template, with reviewUrl required.

Action links are HMAC-signed with a new INTAKE_ACTION_SECRET over (draftId, action, expiry), verified with a timing-safe comparison, good for 30 days.

• Regenerate — back to state='queued'.
• Discard — state='discarded' and the item unpublished; recoverable from the queue, because a one-click destructive action reachable from an inbox should not be final.
• Review & publish — an ordinary deep link into the admin queue, behind authentik like the rest of /admin. It carries no signature and grants nothing.

The two signed actions are deliberately the ones whose worst case is a wasted API call or a recoverable hide. Nothing a single click can do puts an item on the storefront.

The review queue

A new admin screen listing drafts by state: photos, the submitter's note, which link it came from, the drafted copy in editable fields, the price, and Publish / Regenerate / Discard. Publish writes the edited values onto the item and calls the existing mark-available.

The price field carries the weight the schema no longer does, so it is not an ordinary input. It is pre-filled, and it is labelled with where the number came from — the model, or the 80.00 default, or the admin — with anything that is not price_source='admin' marked visibly as unconfirmed. Editing it sets price_source='admin'. Publishing something still marked unconfirmed is allowed, because that is the decision taken, but it says so plainly at the point of publishing rather than after.

Reuses the existing admin item components where they fit rather than growing a second item editor.

Configuration

New variables, added to envValidation.ts and — per #107, enforced by composeEnvironment.test.ts — to both compose files if they go on the required list.

Variable	Requirement
ANTHROPIC_API_KEY	Required when intake is enabled
INTAKE_MODEL	Optional, defaults to claude-sonnet-5
INTAKE_MONTHLY_BUDGET_USD	Optional, defaults to 20
INTAKE_ACTION_SECRET	Required when intake is enabled

Intake is feature-flagged off by default, so none of these become required for an environment that does not run it. PUBLIC_URL is already required alongside SMTP and is what the review links are built from.

Error handling

The through-line: a submission is the only irreplaceable thing here. Photos of a one-of-a-kind object may not be retakeable — the item may not be in the sender's hands any more. Every failure mode below is arranged so the photos survive it.

Failure	Behaviour
API call fails or times out	Retry to three attempts, then state='failed'. Item and photos intact, reviewable with no draft, priced at the default
Budget exhausted	Submission saved, draft skipped, admin still notified, item priced at the default
Malformed model output	Rejected by the schema, counts as a failed attempt
SMTP down	Draft still ready and visible in the queue; the queue, not the email, is the source of truth
Link revoked mid-upload	404. Already-submitted items are unaffected
Restart mid-draft	The cron sweeper re-queues anything stuck in drafting

Testing

• Unit — token hashing and constant-time verification, HMAC signing and expiry, the budget calculation, the prompt builder's handling of an empty note, and the price_source transitions. All pure, in the style of keyByCallerAndEmail and isAllowedRecipient being exported specifically to be tested directly.
• Integration — submission through a valid link creates a pending item with images, a queued draft and the 8000 default; a model suggestion overwrites that price and sets price_source='ai'; a failed or budget-skipped draft leaves the default in place; a revoked link gets 404; a signed action link works once and an expired or tampered one does not. The Anthropic client is stubbed; no test spends money.
• E2E — the submission page, and the queue round trip from draft to published. Assertions scoped to the item under test rather than the whole grid, since the dev database never truncates.

Out of scope

Contributor accounts, a watched NAS folder, email-in submission, multi-item submissions, image editing or cropping, and regenerating with a steer ("try again, warmer"). All are additive later; none change the schema above.