refactor(db): build the item queries through Kysely (#308)

The two queries the builder was ever wanted for. #294 removed interpolation from seven sites by hoisting each fixed-shape query into a named constant; these two genuinely composed their WHERE at run time and could not be fixed that way, which is why they are the last S2077 hotspots. They were safe, and itemFilters.ts spent sixteen lines explaining why — that the clause fragments are literals, that the only things interpolated into them are placeholder indices, and that every value goes onto params. That argument was correct and it was still an argument, guarded by a comment and two tests, on a route reachable without signing in.

All four call sites moved rather than only the two flagged ones. The by-id constants carried no hotspot, but they were built by interpolating the same projection strings the list queries used, so converting only the list queries would have left itemSelect.ts holding a Kysely builder and a raw string that had to produce an identical projection — two spellings to keep in step by hand where the file's own header already warned about one.

The second thing this buys may matter more than the first. pool.query<T> asserts a shape TypeScript never checks against the SQL, which is why that header said the selects and their row types are kept in step by hand and the integration suite was the only thing that caught a drop. The projections are built with jsonArrayFrom now, which emits the same coalesce(json_agg(agg), '[]') they hand-wrote, so the row type follows from the projection and a dropped column is a compile error.

The row types stay hand-written and exported rather than being inferred from the query. Inference would be tidier and is refused deliberately: these are the shape the frontend reads, and a type that silently becomes whatever the query happens to return is a contract that can change without anyone deciding to change it.

The two invariant tests survive and got stronger. They used to inspect the clause strings the builder returned; they now compile the expressions and assert on the SQL Kysely actually emits, with the hostile value appearing in the parameters and nowhere else — the same claim, tested against the real artefact instead of an intermediate one.

Closes #308

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-04 17:00:21 -05:00
co-authored by Claude Opus 5
parent 3248ac656e
commit ec1020891f
5 changed files with 344 additions and 281 deletions
+45 -54
View File
@@ -2,7 +2,9 @@
// filters. Kept apart from the route so the rules can be unit-tested without a // filters. Kept apart from the route so the rules can be unit-tested without a
// database, and so items.ts stays a thin handler. // database, and so items.ts stays a thin handler.
import { Expression, SqlBool, sql } from 'kysely';
import { ItemStatus } from './types'; import { ItemStatus } from './types';
import { ItemContext } from './itemSelect';
export type { ItemStatus }; export type { ItemStatus };
export class FilterError extends Error {} export class FilterError extends Error {}
@@ -59,11 +61,6 @@ export const STOREFRONT_DEFAULT_STATUSES: readonly ItemStatus[] = ['available',
// shape this codebase keeps designing against. // shape this codebase keeps designing against.
export const STOREFRONT_ALL_STATUSES: readonly ItemStatus[] = ['available', 'reserved', 'sold']; export const STOREFRONT_ALL_STATUSES: readonly ItemStatus[] = ['available', 'reserved', 'sold'];
export interface BuiltFilter {
clauses: string[];
params: unknown[];
}
// Deliberately excludes a leading sign and any decimal point: every filter // Deliberately excludes a leading sign and any decimal point: every filter
// value is a non-negative integer (an id, or a price in cents), so '-1' and // value is a non-negative integer (an id, or a price in cents), so '-1' and
// '10.5' are caller mistakes worth surfacing rather than silently coercing. // '10.5' are caller mistakes worth surfacing rather than silently coercing.
@@ -158,7 +155,7 @@ function parseTagIds(value: unknown): number[] {
continue; continue;
} }
const id = parseId(trimmed, 'tags'); const id = parseId(trimmed, 'tags');
// Duplicates would inflate the required-match count in buildItemFilterSql // Duplicates would inflate the required-match count in itemFilterExpressions
// and make the filter match nothing at all. // and make the filter match nothing at all.
if (!tagIds.includes(id)) { if (!tagIds.includes(id)) {
tagIds.push(id); tagIds.push(id);
@@ -236,24 +233,21 @@ export function parseItemFilters(query: Record<string, unknown>): ItemFilters {
return { categoryIds, tagIds, minPriceCents, maxPriceCents, status, favoritesOnly }; return { categoryIds, tagIds, minPriceCents, maxPriceCents, status, favoritesOnly };
} }
// Returns WHERE fragments plus their parameters, with placeholders numbered // Composes the filter clauses as Kysely expressions.
// from `startIndex` so the caller can splice these in after its own params.
// //
// SECURITY INVARIANT, and it is load-bearing. Both callers splice these clauses // This returned `{ clauses: string[]; params: unknown[] }` until #308, and both
// straight into query text — admin.ts as `${ADMIN_ITEM_SELECT} ${where}`, and // callers spliced the clauses straight into query text. The invariant that made
// items.ts as `${PUBLIC_ITEM_SELECT} WHERE ${where}`, which is reachable // that safe — only a placeholder index may ever be interpolated into a clause,
// without signing in. So the only thing that may ever be interpolated into a // never a value — was a sixteen-line comment and two tests standing between an
// string pushed onto `clauses` is a placeholder index: `$${next}`, or // edit and a live injection on a route reachable without signing in.
// `$${next + 1}` in the tags clause. Every value goes onto `params` and is
// bound by the driver. Interpolating a filter value here would be SQL injection
// at both call sites, and `parseItemFilters` refusing malformed input is not
// what prevents it — these literals would be safe with no parser at all.
// //
// Stated here rather than only at the call sites because this is where the rule // It is now a property of the type system. `${value}` inside a Kysely `sql`
// is enforced and where a seventh clause would be added. SonarQube raised S2077 // template emits a bind parameter, never text, and the builder expressions
// on the call sites and they are marked Reviewed/Safe (#180); that marking does // cannot express interpolation at all. The two tests at the bottom of
// not re-raise when this file changes, so this comment and the two tests over // itemFilters.test.ts still exist and now assert against the SQL Kysely
// it are what stand between that edit and a live injection. See #202. // actually emits, which is a stronger claim than the one they used to make.
//
// `startIndex` is gone with the splicing it existed for.
// //
// `favoritesCustomerId` is required rather than optional so a caller has to say // `favoritesCustomerId` is required rather than optional so a caller has to say
// whose favorites it means, even when it means nobody's. Both routes already // whose favorites it means, even when it means nobody's. Both routes already
@@ -261,17 +255,14 @@ export function parseItemFilters(query: Record<string, unknown>): ItemFilters {
// a programming error — but it is here so that a future caller which forgets // a programming error — but it is here so that a future caller which forgets
// the guard fails loudly instead of quietly ignoring the filter and listing the // the guard fails loudly instead of quietly ignoring the filter and listing the
// whole catalogue. // whole catalogue.
export function buildItemFilterSql( export function itemFilterExpressions(
eb: ItemContext,
filters: ItemFilters, filters: ItemFilters,
startIndex: number,
favoritesCustomerId: number | null favoritesCustomerId: number | null
): BuiltFilter { ): Expression<SqlBool>[] {
const clauses: string[] = []; const clauses: Expression<SqlBool>[] = [];
const params: unknown[] = [];
let next = startIndex;
if (filters.categoryIds.length) { if (filters.categoryIds.length) {
params.push(filters.categoryIds);
// Selecting a category means "and everything filed beneath it", so walk the // Selecting a category means "and everything filed beneath it", so walk the
// tree down from each chosen node. A recursive CTE keeps the tree // tree down from each chosen node. A recursive CTE keeps the tree
// un-denormalized: reparenting stays a single UPDATE with no stored paths // un-denormalized: reparenting stays a single UPDATE with no stored paths
@@ -281,61 +272,61 @@ export function buildItemFilterSql(
// walked in the same recursion. That also gives the OR for free: the union // walked in the same recursion. That also gives the OR for free: the union
// of the subtrees is exactly "filed under any of these", and an item filed // of the subtrees is exactly "filed under any of these", and an item filed
// under two selected branches appears once because IN is a set test. // under two selected branches appears once because IN is a set test.
clauses.push(`i.category_id IN ( //
// Still a `sql` template, because the builder expresses a recursive CTE no
// better than this does. `${filters.categoryIds}` is one bind parameter
// holding the whole array — not a placeholder list — which is why no
// sql.param() ceremony appears here. See src/db-kysely/CONVENTIONS.md.
clauses.push(sql<SqlBool>`i.category_id IN (
WITH RECURSIVE subtree AS ( WITH RECURSIVE subtree AS (
SELECT id FROM categories WHERE id = ANY($${next}::int[]) SELECT id FROM categories WHERE id = ANY(${filters.categoryIds}::int[])
UNION ALL UNION ALL
SELECT c.id FROM categories c JOIN subtree s ON c.parent_id = s.id SELECT c.id FROM categories c JOIN subtree s ON c.parent_id = s.id
) )
SELECT id FROM subtree SELECT id FROM subtree
)`); )`);
next++;
} }
if (filters.tagIds.length) { if (filters.tagIds.length) {
params.push(filters.tagIds, filters.tagIds.length);
// AND, not OR: the item must carry every selected tag. Matching with // AND, not OR: the item must carry every selected tag. Matching with
// `tag_id = ANY(...)` alone would return items holding just one of them, so // `tag_id = ANY(...)` alone would return items holding just one of them, so
// the count of matched rows has to equal the number requested. // the count of matched rows has to equal the number requested.
clauses.push( clauses.push(sql<SqlBool>`(SELECT COUNT(*) FROM item_tags it
`(SELECT COUNT(*) FROM item_tags it WHERE it.item_id = i.id AND it.tag_id = ANY(${filters.tagIds}::int[])) = ${filters.tagIds.length}`);
WHERE it.item_id = i.id AND it.tag_id = ANY($${next}::int[])) = $${next + 1}`
);
next += 2;
} }
if (filters.minPriceCents !== null) { if (filters.minPriceCents !== null) {
params.push(filters.minPriceCents); clauses.push(eb('i.price_cents', '>=', filters.minPriceCents));
clauses.push(`i.price_cents >= $${next}`);
next++;
} }
if (filters.maxPriceCents !== null) { if (filters.maxPriceCents !== null) {
params.push(filters.maxPriceCents); clauses.push(eb('i.price_cents', '<=', filters.maxPriceCents));
clauses.push(`i.price_cents <= $${next}`);
next++;
} }
if (filters.status !== null) { if (filters.status !== null) {
params.push(filters.status); // `in` replaces the `= ANY($n::text[])` this used to build. Kysely emits
// ANY rather than equality, so one status and several use the same clause. // the placeholder list itself, so one status and several use the same
// The ::text[] cast is explicit because `status` is a text column and the // expression and the explicit ::text[] cast is no longer needed.
// driver would otherwise have to infer the array's element type. clauses.push(eb('i.status', 'in', filters.status));
clauses.push(`i.status = ANY($${next}::text[])`);
next++;
} }
if (filters.favoritesOnly) { if (filters.favoritesOnly) {
if (favoritesCustomerId === null) { if (favoritesCustomerId === null) {
throw new Error('favorites filter requires a customer id'); throw new Error('favorites filter requires a customer id');
} }
params.push(favoritesCustomerId);
// EXISTS rather than a join: an item is favorited by a customer at most // EXISTS rather than a join: an item is favorited by a customer at most
// once, but joining would still risk multiplying rows if that ever changed, // once, but joining would still risk multiplying rows if that ever changed,
// and this reads as the membership test it is. // and this reads as the membership test it is.
clauses.push(`EXISTS (SELECT 1 FROM favorites f WHERE f.item_id = i.id AND f.customer_id = $${next})`); clauses.push(
next++; eb.exists(
eb
.selectFrom('favorites as f')
.select('f.item_id')
.whereRef('f.item_id', '=', 'i.id')
.where('f.customer_id', '=', favoritesCustomerId)
)
);
} }
return { clauses, params }; return clauses;
} }
+98 -69
View File
@@ -1,95 +1,124 @@
// Shared item SELECT shapes for the public and admin routes, and the row types // Shared item query shapes for the public and admin routes, and the row types
// they return. // they return.
// //
// The types live here rather than in types.ts because they describe a // The types live here rather than in types.ts because they describe a
// projection, not a table. ADMIN_ITEM_SELECT takes `i.*` and PUBLIC_ITEM_SELECT // projection, not a table. adminItemQuery takes every column of items and
// names its columns so the storefront never sees paypal_order_id or // publicItemQuery names its columns, so the storefront never sees
// reserved_until — typing both as "an items row" would quietly re-admit exactly // paypal_order_id or reserved_until — typing both as "an items row" would
// the columns that select was written to exclude. // quietly re-admit exactly the columns that projection was written to exclude.
// //
// KEPT IN STEP BY HAND. `pool.query<T>` asserts a shape; it does not check the // These were SQL string constants until #308. They had to be kept in step with
// SQL, which TypeScript never reads. Dropping a column from a select below // their row types by hand, because `pool.query<T>` asserts a shape and never
// without dropping it from its type compiles cleanly and every read of it goes // checks it against the SQL, so dropping a column from a select without
// on type-checking while being undefined at runtime. The integration suite is // dropping it from its type compiled cleanly and went undefined at run time —
// the only thing that catches that, because it runs these queries against a // and only the integration suite ever caught it. Built through Kysely, that is
// real schema. Change a select and its type together. // a compile error, because the row type now follows from the projection.
// //
// Images and tags are pulled as scalar subqueries rather than LEFT JOIN + // Images and tags are pulled as aggregate subqueries rather than LEFT JOIN +
// GROUP BY. Joining two one-to-many relations in the same query multiplies // GROUP BY. Joining two one-to-many relations in the same query multiplies
// their rows together — an item with 2 images and 3 tags would aggregate 6 // their rows together — an item with 2 images and 3 tags would aggregate 6
// rows, silently repeating every image three times. Subqueries keep each // rows, silently repeating every image three times. Subqueries keep each
// aggregate independent and drop the GROUP BY entirely. // aggregate independent and drop the GROUP BY entirely. `jsonArrayFrom` emits
// `coalesce(json_agg(agg), '[]')`, which is what these hand-wrote before.
import { ExpressionBuilder } from 'kysely';
import { jsonArrayFrom } from 'kysely/helpers/postgres';
import { db } from './db';
import { DB } from './db-kysely/schema';
import { ItemStatus, ItemImage, ItemTag } from './types'; import { ItemStatus, ItemImage, ItemTag } from './types';
const IMAGES_SUBQUERY = ` /**
COALESCE(( * The aliases every item query and every filter clause is written against.
SELECT json_agg(json_build_object('id', img.id, 'image_path', img.image_path, 'sort_order', img.sort_order) *
ORDER BY img.sort_order) * `i` and `c` are kept from the SQL these replaced. Not because short names are
FROM item_images img * better, but because the filter clauses, the subquery correlations and the
WHERE img.item_id = i.id * ORDER BY all reference them, and renaming them in the same change that moved
), '[]') AS images`; * the builder would have made the diff unreadable against the SQL it replaces.
*/
export type ItemContext = ExpressionBuilder<
DB & { i: DB['items']; c: DB['categories'] },
'i' | 'c'
>;
/** The public image fields. Correlated to the outer item by `whereRef`. */
function imagesFor(eb: ItemContext) {
return jsonArrayFrom(
eb
.selectFrom('item_images as img')
.select(['img.id', 'img.image_path', 'img.sort_order'])
.whereRef('img.item_id', '=', 'i.id')
.orderBy('img.sort_order')
).as('images');
}
/** /**
* Admin-only images, carrying `original_image_path` alongside the public * Admin-only images, carrying `original_image_path` alongside the public
* fields — the field the inventory screen needs to know whether a photo has a * fields — the field the inventory screen needs to know whether a photo has a
* cut-out to restore (#293). * cut-out to restore (#293).
* *
* A separate subquery rather than adding the column to `IMAGES_SUBQUERY` * A separate function rather than a flag on `imagesFor`, for the same reason
* itself, for the same reason `PUBLIC_ITEM_SELECT` names its columns instead * `publicItemQuery` names its columns instead of taking them all: an original
* of using `i.*`: an original filename is internal nobody's business on the * filename is internal, nobody's business on the storefront, and a boolean in
* storefront — and folding it into the one subquery both selects share would * the middle of the thing that keeps it off the public API is one edit away
* put it in every public item response too. * from being passed wrongly.
*/ */
const ADMIN_IMAGES_SUBQUERY = ` function adminImagesFor(eb: ItemContext) {
COALESCE(( return jsonArrayFrom(
SELECT json_agg(json_build_object('id', img.id, 'image_path', img.image_path, 'sort_order', img.sort_order, eb
'original_image_path', img.original_image_path) .selectFrom('item_images as img')
ORDER BY img.sort_order) .select(['img.id', 'img.image_path', 'img.sort_order', 'img.original_image_path'])
FROM item_images img .whereRef('img.item_id', '=', 'i.id')
WHERE img.item_id = i.id .orderBy('img.sort_order')
), '[]') AS images`; ).as('images');
}
const TAGS_SUBQUERY = ` function tagsFor(eb: ItemContext) {
COALESCE(( return jsonArrayFrom(
SELECT json_agg(json_build_object('id', t.id, 'name', t.name, 'color', t.color) ORDER BY t.name) eb
FROM item_tags it .selectFrom('item_tags as it')
JOIN tags t ON t.id = it.tag_id .innerJoin('tags as t', 't.id', 'it.tag_id')
WHERE it.item_id = i.id .select(['t.id', 't.name', 't.color'])
), '[]') AS tags`; .whereRef('it.item_id', '=', 'i.id')
.orderBy('t.name')
const FROM_CLAUSE = ` ).as('tags');
FROM items i }
LEFT JOIN categories c ON c.id = i.category_id`;
// The storefront gets an explicit column list — it has no business seeing
// paypal_order_id or reserved_until.
export const PUBLIC_ITEM_SELECT = `
SELECT i.id, i.name, i.description, i.price_cents, i.status, i.created_at, i.category_id,
c.name AS category_name,
${IMAGES_SUBQUERY},
${TAGS_SUBQUERY}
${FROM_CLAUSE}`;
export const ADMIN_ITEM_SELECT = `
SELECT i.*,
c.name AS category_name,
${ADMIN_IMAGES_SUBQUERY},
${TAGS_SUBQUERY}
${FROM_CLAUSE}`;
/** /**
* One admin item, by id — the whole query, not a fragment. * The storefront's projection — an explicit column list, because it has no
* business seeing paypal_order_id or reserved_until.
* *
* A named constant rather than `${ADMIN_ITEM_SELECT} WHERE i.id = $1` written * A function rather than a constant so each caller gets a fresh builder. Kysely
* at each call, so no query call site interpolates anything at all. The id was * builders are immutable, so sharing one would be safe, but a function makes it
* always bound as $1 and never reached the query text, but S2077 fires on the * obvious that adding a `where` does not affect anyone else.
* template literal rather than on the value, because the rule cannot tell a
* module constant from a request field — and neither, at a glance, can a
* reader. Hoisting it makes the property structural instead of an assertion
* somebody has to re-make every time the line moves. See #294.
*/ */
export const ADMIN_ITEM_BY_ID = `${ADMIN_ITEM_SELECT} WHERE i.id = $1`; export function publicItemQuery() {
return db
.selectFrom('items as i')
.leftJoin('categories as c', 'c.id', 'i.category_id')
.select([
'i.id',
'i.name',
'i.description',
'i.price_cents',
'i.status',
'i.created_at',
'i.category_id',
'c.name as category_name'
])
.select(imagesFor)
.select(tagsFor);
}
/** The admin projection — every item column, plus the admin image fields. */
export function adminItemQuery() {
return db
.selectFrom('items as i')
.leftJoin('categories as c', 'c.id', 'i.category_id')
.selectAll('i')
.select('c.name as category_name')
.select(adminImagesFor)
.select(tagsFor);
}
/** The columns every item select returns, whichever of the two it is. */ /** The columns every item select returns, whichever of the two it is. */
interface ItemRowBase { interface ItemRowBase {
+20 -19
View File
@@ -1,10 +1,10 @@
import { Router, Request, Response } from 'express'; import { Router, Request, Response } from 'express';
import { PoolClient } from 'pg'; import { PoolClient } from 'pg';
import { pool, requireRow } from '../db'; import { pool, requireRow } from '../db';
import { ADMIN_ITEM_SELECT, ADMIN_ITEM_BY_ID, AdminItemRow, ItemRecord } from '../itemSelect'; import { adminItemQuery, AdminItemRow, ItemRecord } from '../itemSelect';
import { ItemStatus } from '../types'; import { ItemStatus } from '../types';
import { asyncRoute } from '../asyncRoute'; import { asyncRoute } from '../asyncRoute';
import { parseItemFilters, buildItemFilterSql, FilterError } from '../itemFilters'; import { parseItemFilters, itemFilterExpressions, FilterError } from '../itemFilters';
import { readId, tagColorFor } from '../utils'; import { readId, tagColorFor } from '../utils';
import { notifyFavoritersOfSale, notifyFavoritersOfRemoval, collectFavoriteRecipients } from '../favoriteAlerts'; import { notifyFavoritersOfSale, notifyFavoritersOfRemoval, collectFavoriteRecipients } from '../favoriteAlerts';
import { removeBackgroundsForItem, restoreOriginalsForItem } from '../intake/backgroundRemoval'; import { removeBackgroundsForItem, restoreOriginalsForItem } from '../intake/backgroundRemoval';
@@ -132,21 +132,22 @@ router.get('/items', asyncRoute(async (req: Request, res: Response) => {
return res.status(400).json({ error: 'favorites is not a valid inventory filter' }); return res.status(400).json({ error: 'favorites is not a valid inventory filter' });
} }
// S2077 flags every query below that assembles its SQL as a template literal, // No interpolation, and nothing to argue about. Until #308 this assembled
// and this is the one where that is more than a formality: `where` really is // `${ADMIN_ITEM_SELECT} ${where}` from clauses composed at run time, and
// built at run time. What makes it safe is that buildItemFilterSql composes // sixteen lines in itemFilters.ts explained why that was safe. The clauses
// only string literals written in itemFilters.ts. The only interpolations // are Kysely expressions now: a value cannot reach the SQL text, because the
// inside any of them are placeholder indices — `$${next}`, and `$${next + 1}` // types do not let it.
// in the tags clause — numbers, seeded from the startIndex argument and // `.$castTo` narrows `status` from the schema mirror's generic `string` (the
// incremented locally. Neither is ever derived from a filter value. // column is a CHECK-constrained text column, not a native Postgres enum, so
// // kysely-codegen has no literal union to give it) to the app-level
// So a caller chooses which of six fixed fragments are joined, and supplies // `ItemStatus` the CHECK constraint actually enforces. Every other field on
// every value in `params`, and neither of those becomes SQL. parseItemFilters // AdminItemRow already matches the projection without a cast.
// rejects malformed input above, but that is defence in depth rather than the const rows: AdminItemRow[] = await adminItemQuery()
// reason this holds — the clause literals would be safe without it. .where((eb) => eb.and(itemFilterExpressions(eb, filters, null)))
const { clauses, params } = buildItemFilterSql(filters, 1, null); .orderBy('i.created_at', 'desc')
const where = clauses.length ? `WHERE ${clauses.join(' AND ')}` : ''; .$castTo<AdminItemRow>()
const { rows } = await pool.query<AdminItemRow>(`${ADMIN_ITEM_SELECT} ${where} ORDER BY i.created_at DESC`, params); .execute();
res.json(rows); res.json(rows);
})); }));
@@ -175,7 +176,7 @@ router.post('/items', uploadImages, asyncRoute(async (req: Request, res: Respons
// is bound as $1. It always was bound — what changed is that a reader no // is bound as $1. It always was bound — what changed is that a reader no
// longer has to check that the interpolated half carries no caller data, // longer has to check that the interpolated half carries no caller data,
// because there is no interpolated half. See #294. // because there is no interpolated half. See #294.
const { rows: full } = await pool.query<AdminItemRow>(ADMIN_ITEM_BY_ID, [item.id]); const full = await adminItemQuery().where('i.id', '=', item.id).execute();
res.json(requireRow(full, 'the item just inserted')); res.json(requireRow(full, 'the item just inserted'));
} catch (err) { } catch (err) {
await client.query('ROLLBACK'); await client.query('ROLLBACK');
@@ -229,7 +230,7 @@ router.put('/items/:id', uploadImages, asyncRoute(async (req: Request, res: Resp
// The same constant as the create route above. itemId is caller-controlled // The same constant as the create route above. itemId is caller-controlled
// and goes through the driver as a bound parameter; it never reaches the // and goes through the driver as a bound parameter; it never reaches the
// query text. // query text.
const { rows: full } = await pool.query<AdminItemRow>(ADMIN_ITEM_BY_ID, [itemId]); const full = await adminItemQuery().where('i.id', '=', itemId).execute();
// The create route beside this one has always used requireRow here. This // The create route beside this one has always used requireRow here. This
// one did not, so an UPDATE matching nothing committed happily, the SELECT // one did not, so an UPDATE matching nothing committed happily, the SELECT
// returned nothing, and the caller got 200 with an empty body — a success // returned nothing, and the caller got 200 with an empty body — a success
+32 -32
View File
@@ -1,30 +1,28 @@
import { Router, Request, Response } from 'express'; import { Router, Request, Response } from 'express';
import { pool } from '../db';
import { asyncRoute } from '../asyncRoute'; import { asyncRoute } from '../asyncRoute';
import { PUBLIC_ITEM_SELECT, PublicItemRow } from '../itemSelect'; import { publicItemQuery, PublicItemRow, ItemContext } from '../itemSelect';
import { import {
parseItemFilters, parseItemFilters,
buildItemFilterSql, itemFilterExpressions,
FilterError, FilterError,
NON_PUBLIC_STATUSES, NON_PUBLIC_STATUSES,
STOREFRONT_DEFAULT_STATUSES, STOREFRONT_DEFAULT_STATUSES,
STOREFRONT_ALL_STATUSES STOREFRONT_ALL_STATUSES
} from '../itemFilters'; } from '../itemFilters';
// Applied to every public read, unconditionally. This route has never had a
// status filter of its own — sold items are listed and rendered with a Sold
// badge on purpose — so hiding pending items cannot be expressed as one more
// optional filter. It has to be a clause the caller cannot opt out of.
const EXCLUDE_PENDING = `i.status <> 'pending'`;
/** /**
* One public item, by id — the whole query rather than a fragment. * Pending items are excluded everywhere, not only from the list. A pending item
* that stayed fetchable by id would be hidden from the catalogue and still
* reachable by anyone who guessed or kept a link.
* *
* Built once here so the call site interpolates nothing. Both halves were * An expression rather than the SQL literal this was until #308, so it composes
* always constants and the id was always bound as $1, but a template literal at * with the filter clauses through `eb.and` instead of being joined into a
* a query call is a thing a reader has to verify rather than see. See #294. * string. That join used to need its own argument about why AND could not
* weaken it; `and` cannot re-associate anything.
*/ */
const PUBLIC_ITEM_BY_ID = `${PUBLIC_ITEM_SELECT} WHERE i.id = $1 AND ${EXCLUDE_PENDING}`; function notPending(eb: ItemContext) {
return eb('i.status', '!=', 'pending');
}
const router = Router(); const router = Router();
@@ -81,28 +79,30 @@ router.get('/', asyncRoute(async (req: Request, res: Response) => {
status: filters.status ?? [...defaultStatuses] status: filters.status ?? [...defaultStatuses]
}; };
const { clauses, params } = buildItemFilterSql(effectiveFilters, 1, req.customerId ?? null); // `.$castTo` narrows `status` from the schema mirror's generic `string` (the
// The same construct SonarQube flagged as S2077 in admin.ts and which is // column is a CHECK-constrained text column, not a native Postgres enum, so
// marked Reviewed/Safe there (#180) — and this is the copy reachable without // kysely-codegen has no literal union to give it) to the app-level
// signing in, so it is worth saying here too rather than relying on the // `ItemStatus` the CHECK constraint actually enforces. Every other field on
// reader having seen the other one. It holds for the same reason: the clauses // PublicItemRow already matches the projection without a cast.
// are literals from buildItemFilterSql carrying only placeholder indices, and const rows: PublicItemRow[] = await publicItemQuery()
// EXCLUDE_PENDING is a module constant. Joining with AND cannot weaken .where((eb) =>
// EXCLUDE_PENDING either, because no fragment contains a top-level OR for the eb.and([
// join to re-associate against. notPending(eb),
const where = [EXCLUDE_PENDING, ...clauses].join(' AND '); ...itemFilterExpressions(eb, effectiveFilters, req.customerId ?? null)
const { rows } = await pool.query<PublicItemRow>( ])
`${PUBLIC_ITEM_SELECT} WHERE ${where} ORDER BY i.created_at DESC`, )
params .orderBy('i.created_at', 'desc')
); .$castTo<PublicItemRow>()
.execute();
res.json(rows); res.json(rows);
})); }));
router.get('/:id', asyncRoute(async (req: Request, res: Response) => { router.get('/:id', asyncRoute(async (req: Request, res: Response) => {
// Excluded here too, not only from the list. A pending item that stayed const rows = await publicItemQuery()
// fetchable by id would be hidden from the catalogue and still reachable by .where('i.id', '=', Number(req.params.id))
// anyone who guessed or kept a link. .where((eb) => notPending(eb))
const { rows } = await pool.query<PublicItemRow>(PUBLIC_ITEM_BY_ID, [req.params.id]); .execute();
if (!rows.length) return res.status(404).json({ error: 'not found' }); if (!rows.length) return res.status(404).json({ error: 'not found' });
res.json(rows[0]); res.json(rows[0]);
})); }));
+135 -93
View File
@@ -1,4 +1,6 @@
import { parseItemFilters, FilterError, buildItemFilterSql } from '../../src/itemFilters'; import { parseItemFilters, FilterError } from '../../src/itemFilters';
import { db } from '../../src/db';
import { itemFilterExpressions, ItemFilters } from '../../src/itemFilters';
describe('parseItemFilters', () => { describe('parseItemFilters', () => {
it('returns empty filters for an empty query', () => { it('returns empty filters for an empty query', () => {
@@ -170,138 +172,178 @@ describe('parseItemFilters', () => {
}); });
}); });
describe('buildItemFilterSql', () => { /**
it('produces no clauses and no params when nothing is filtered', () => { * Compiles the filter clauses on their own, with no projection around them.
const built = buildItemFilterSql(parseItemFilters({}), 1, null); *
expect(built.clauses).toEqual([]); * The expressions are what this file is about, and Kysely compiles without a
expect(built.params).toEqual([]); * connection — so these assert on the SQL and parameters actually emitted,
* rather than on the intermediate strings the old builder returned. That is a
* stronger claim than the one these tests used to make.
*/
function compileFilters(filters: ItemFilters, customerId: number | null = null) {
// The same `items as i` + `categories as c` shape both real queries use, so
// the expression builder handed to the callback is exactly the ItemContext
// the filters are written against. Building a narrower query here would need
// a cast, and a cast in the test would be testing the cast.
const { sql, parameters } = db
.selectFrom('items as i')
.leftJoin('categories as c', 'c.id', 'i.category_id')
.select('i.id')
.where((eb) => eb.and(itemFilterExpressions(eb, filters, customerId)))
.compile();
return { sql, parameters: [...parameters] };
}
const NO_FILTERS = {
categoryIds: [],
tagIds: [],
minPriceCents: null,
maxPriceCents: null,
status: null,
favoritesOnly: false
};
describe('itemFilterExpressions', () => {
it('adds no condition when nothing is filtered', () => {
const { sql, parameters } = compileFilters(NO_FILTERS);
// Not "no WHERE at all" as originally assumed: Kysely 0.28's `eb.and([])`
// compiles an empty conjunction to the truism `where 1 = 1` rather than
// omitting the clause (see parseFilterList in
// kysely/dist/cjs/parser/binary-operation-parser.js). That still matches
// every row, so it is the same "no filter" behaviour the old
// `clauses.length ? ... : ''` gave — the literal SQL text just differs
// from what was assumed here, which is what this assertion now checks.
expect(sql).toContain('where 1 = 1');
expect(parameters).toEqual([]);
}); });
it('matches a category and all of its descendants', () => { it('matches a category and all of its descendants', () => {
const built = buildItemFilterSql(parseItemFilters({ category: '4' }), 1, null); const { sql, parameters } = compileFilters({ ...NO_FILTERS, categoryIds: [4] });
expect(built.clauses.join(' ')).toContain('RECURSIVE'); expect(sql).toContain('WITH RECURSIVE subtree');
// One array parameter rather than one id: the CTE is seeded with ANY so expect(parameters).toEqual([[4]]);
// several selected roots are walked in the same recursion.
expect(built.params).toEqual([[4]]);
}); });
it('seeds the descendant walk with every selected category', () => { // One bind parameter holding the whole array, not a placeholder list. This is
const built = buildItemFilterSql(parseItemFilters({ category: '4,9' }), 1, null); // the property that made the array trap in the previous builder impossible
const sql = built.clauses.join(' '); // here — see #297 and src/db-kysely/CONVENTIONS.md.
expect(sql).toContain('RECURSIVE'); it('seeds the descendant walk with every selected category, as one parameter', () => {
// ANY over the seeds is what makes several categories combine as OR: the const { parameters } = compileFilters({ ...NO_FILTERS, categoryIds: [4, 9] });
// result is the union of their subtrees. expect(parameters).toEqual([[4, 9]]);
expect(sql).toContain('= ANY($1::int[])');
expect(built.params).toEqual([[4, 9]]);
}); });
it('requires every listed tag rather than any of them', () => { it('requires every listed tag rather than any of them', () => {
const built = buildItemFilterSql(parseItemFilters({ tags: '1,2' }), 1, null); const { sql, parameters } = compileFilters({ ...NO_FILTERS, tagIds: [2, 5] });
// The count of matched tag rows must equal the number of tags requested — expect(sql).toContain('SELECT COUNT(*) FROM item_tags');
// an ANY/IN match alone would return items carrying just one of them. expect(parameters).toEqual([[2, 5], 2]);
expect(built.clauses.join(' ')).toContain('COUNT(*)');
expect(built.params).toEqual([[1, 2], 2]);
}); });
it('numbers placeholders from the given starting index', () => { it('filters on a price range', () => {
const built = buildItemFilterSql(parseItemFilters({ category: '4' }), 3, null); const { parameters } = compileFilters({
expect(built.clauses.join(' ')).toContain('$3'); ...NO_FILTERS,
minPriceCents: 1000,
maxPriceCents: 5000
});
expect(parameters).toEqual([1000, 5000]);
}); });
it('filters on status', () => { it('filters on several statuses with one expression', () => {
const built = buildItemFilterSql(parseItemFilters({ status: 'reserved' }), 1, null); const { sql, parameters } = compileFilters({
expect(built.clauses.join(' ')).toContain('i.status = ANY'); ...NO_FILTERS,
expect(built.params).toEqual([['reserved']]); status: ['available', 'reserved']
}); });
expect(sql).toContain('"i"."status" in');
// One clause for one status and for several, which is the whole reason the expect(parameters).toEqual(['available', 'reserved']);
// filter was generalised rather than joined by a second dimension.
it('filters on several statuses with the same single clause', () => {
const built = buildItemFilterSql(parseItemFilters({ status: 'available,reserved' }), 1, null);
expect(built.clauses).toHaveLength(1);
expect(built.clauses[0]).toContain('i.status = ANY');
expect(built.params).toEqual([['available', 'reserved']]);
}); });
it('restricts to the favorites of the given customer', () => { it('restricts to the favorites of the given customer', () => {
const built = buildItemFilterSql(parseItemFilters({ favorites: '1' }), 1, 42); const { sql, parameters } = compileFilters({ ...NO_FILTERS, favoritesOnly: true }, 7);
expect(built.clauses.join(' ')).toContain('EXISTS'); expect(sql).toContain('exists');
expect(built.clauses.join(' ')).toContain('favorites f'); expect(parameters).toEqual([7]);
expect(built.params).toEqual([42]);
}); });
it('does not restrict to favorites when the flag is off, even given a customer', () => { it('does not restrict to favorites when the flag is off, even given a customer', () => {
const built = buildItemFilterSql(parseItemFilters({}), 1, 42); const { sql, parameters } = compileFilters(NO_FILTERS, 7);
expect(built.clauses).toEqual([]); expect(sql).not.toContain('exists');
expect(built.params).toEqual([]); expect(parameters).toEqual([]);
}); });
// Both routes reject this before reaching the builder, so it can only happen
// through a new caller that forgot to. Failing loudly beats dropping the
// clause and returning the whole catalogue as if it were someone's favorites.
it('throws rather than ignore a favorites filter with no customer', () => { it('throws rather than ignore a favorites filter with no customer', () => {
expect(() => buildItemFilterSql(parseItemFilters({ favorites: '1' }), 1, null)).toThrow(); expect(() => compileFilters({ ...NO_FILTERS, favoritesOnly: true }, null)).toThrow(
}); /favorites filter requires a customer id/
it('continues numbering across multiple filters', () => {
const built = buildItemFilterSql(
parseItemFilters({ category: '4', min_price: '100', max_price: '900' }),
1,
null
); );
expect(built.params).toEqual([[4], 100, 900]); });
const sql = built.clauses.join(' ');
expect(sql).toContain('$1'); it('composes several filters together', () => {
expect(sql).toContain('$2'); const { parameters } = compileFilters(
expect(sql).toContain('$3'); {
categoryIds: [4],
tagIds: [2],
minPriceCents: 1000,
maxPriceCents: null,
status: ['available'],
favoritesOnly: true
},
7
);
expect(parameters).toEqual([[4], [2], 1, 1000, 'available', 7]);
}); });
}); });
// Both callers splice these clauses straight into query text, so a value // The invariant, and it is load-bearing: the storefront call site is reachable
// reaching the clause string is SQL injection rather than a style problem. The // without signing in, so a filter value reaching the SQL text is SQL injection
// comment on buildItemFilterSql says so; these two make it fail a build instead // rather than a style problem. These two made that fail a build rather than
// of relying on someone reading it. See #202, and #180 for the S2077 review. // relying on someone reading a comment, and they still do — but they now check
describe('buildItemFilterSql keeps every value out of the SQL text', () => { // the SQL Kysely actually emits rather than the strings the old builder
// Deliberately built by hand rather than through parseItemFilters, because // returned. See #202, #180 for the S2077 review, and #308 for the conversion.
// the claim is that the clause literals are safe with no parser at all. These describe('itemFilterExpressions keeps every value out of the SQL text', () => {
// values could never survive parsing, which is the point: the parser is // Built by hand rather than through parseItemFilters, because the claim is
// defence in depth, not the reason this holds. // that the expressions are safe with no parser at all. These values could
// never survive parsing, which is the point: the parser is defence in depth,
// not the reason this holds.
const HOSTILE = "1); DROP TABLE items; --"; const HOSTILE = "1); DROP TABLE items; --";
const hostileFilters = {
it('never lets a filter value reach the SQL, even one the parser would reject', () => {
const { sql, parameters } = compileFilters(
{
categoryIds: [HOSTILE], categoryIds: [HOSTILE],
tagIds: [HOSTILE], tagIds: [HOSTILE],
minPriceCents: HOSTILE, minPriceCents: HOSTILE,
maxPriceCents: HOSTILE, maxPriceCents: HOSTILE,
status: [HOSTILE], status: [HOSTILE],
favoritesOnly: true favoritesOnly: true
} as unknown as Parameters<typeof buildItemFilterSql>[0]; } as unknown as ItemFilters,
HOSTILE as unknown as number
);
it('never lets a filter value reach a clause, even one the parser would reject', () => {
const built = buildItemFilterSql(hostileFilters, 1, HOSTILE as unknown as number);
const sql = built.clauses.join(' AND ');
expect(sql).not.toContain(HOSTILE);
expect(sql).not.toContain('DROP TABLE'); expect(sql).not.toContain('DROP TABLE');
// Every value still arrives, bound, where it can do nothing. expect(JSON.stringify(parameters)).toContain('DROP TABLE');
expect(built.params).toContain(HOSTILE);
}); });
// The structural version of the same claim, and the one that catches a value
// which happens not to look hostile: the SQL text must not depend on the
// values at all. Two disjoint sets of inputs, byte-identical clauses.
it('produces byte-identical SQL for two completely different filter sets', () => { it('produces byte-identical SQL for two completely different filter sets', () => {
const a = buildItemFilterSql( const first = compileFilters(
parseItemFilters({ category: '4', tags: '7,8', min_price: '100', max_price: '900', status: 'sold' }), {
1, categoryIds: [1],
42 tagIds: [2],
minPriceCents: 3,
maxPriceCents: 4,
status: ['available'],
favoritesOnly: true
},
5
); );
const b = buildItemFilterSql( const second = compileFilters(
parseItemFilters({ category: '99', tags: '11,12', min_price: '5', max_price: '6', status: 'available' }), {
1, categoryIds: [99],
7 tagIds: [98],
minPriceCents: 97,
maxPriceCents: 96,
status: ['sold'],
favoritesOnly: true
},
95
); );
expect(a.clauses).toEqual(b.clauses); expect(first.sql).toBe(second.sql);
expect(a.params).not.toEqual(b.params); expect(first.parameters).not.toEqual(second.parameters);
}); });
}); });