bermudalamb/redefined-designs
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects 3 Releases Wiki Activity

refactor(db): build the item queries through Kysely (#308)

Browse Source 
The two queries the builder was ever wanted for. #294 removed interpolation from seven sites by hoisting each fixed-shape query into a named constant; these two genuinely composed their WHERE at run time and could not be fixed that way, which is why they are the last S2077 hotspots. They were safe, and itemFilters.ts spent sixteen lines explaining why — that the clause fragments are literals, that the only things interpolated into them are placeholder indices, and that every value goes onto params. That argument was correct and it was still an argument, guarded by a comment and two tests, on a route reachable without signing in.

All four call sites moved rather than only the two flagged ones. The by-id constants carried no hotspot, but they were built by interpolating the same projection strings the list queries used, so converting only the list queries would have left itemSelect.ts holding a Kysely builder and a raw string that had to produce an identical projection — two spellings to keep in step by hand where the file's own header already warned about one.

The second thing this buys may matter more than the first. pool.query<T> asserts a shape TypeScript never checks against the SQL, which is why that header said the selects and their row types are kept in step by hand and the integration suite was the only thing that caught a drop. The projections are built with jsonArrayFrom now, which emits the same coalesce(json_agg(agg), '[]') they hand-wrote, so the row type follows from the projection and a dropped column is a compile error.

The row types stay hand-written and exported rather than being inferred from the query. Inference would be tidier and is refused deliberately: these are the shape the frontend reads, and a type that silently becomes whatever the query happens to return is a contract that can change without anyone deciding to change it.

The two invariant tests survive and got stronger. They used to inspect the clause strings the builder returned; they now compile the expressions and assert on the SQL Kysely actually emits, with the hostile value appearing in the parameters and nowhere else — the same claim, tested against the real artefact instead of an intermediate one.

Closes #308

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
bermudalamb
2026-09-04 17:00:21 -05:00
co-authored by Claude Opus 5
parent 3248ac656e
commit ec1020891f
5 changed files with 344 additions and 281 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/tests/unit/itemFilters.test.ts
+149 -107
View File
@@ -1,4 +1,6 @@
import { parseItemFilters, FilterError, buildItemFilterSql } from '../../src/itemFilters';
import { parseItemFilters, FilterError } from '../../src/itemFilters';
import { db } from '../../src/db';
import { itemFilterExpressions, ItemFilters } from '../../src/itemFilters';
describe('parseItemFilters', () => {
it('returns empty filters for an empty query', () => {
@@ -170,138 +172,178 @@ describe('parseItemFilters', () => {
});
});
describe('buildItemFilterSql', () => {
it('produces no clauses and no params when nothing is filtered', () => {
const built = buildItemFilterSql(parseItemFilters({}), 1, null);
expect(built.clauses).toEqual([]);
expect(built.params).toEqual([]);
/**
* Compiles the filter clauses on their own, with no projection around them.
*
* The expressions are what this file is about, and Kysely compiles without a
* connection — so these assert on the SQL and parameters actually emitted,
* rather than on the intermediate strings the old builder returned. That is a
* stronger claim than the one these tests used to make.
*/
function compileFilters(filters: ItemFilters, customerId: number | null = null) {
// The same `items as i` + `categories as c` shape both real queries use, so
// the expression builder handed to the callback is exactly the ItemContext
// the filters are written against. Building a narrower query here would need
// a cast, and a cast in the test would be testing the cast.
const { sql, parameters } = db
.selectFrom('items as i')
.leftJoin('categories as c', 'c.id', 'i.category_id')
.select('i.id')
.where((eb) => eb.and(itemFilterExpressions(eb, filters, customerId)))
.compile();
return { sql, parameters: [...parameters] };
}
const NO_FILTERS = {
categoryIds: [],
tagIds: [],
minPriceCents: null,
maxPriceCents: null,
status: null,
favoritesOnly: false
};
describe('itemFilterExpressions', () => {
it('adds no condition when nothing is filtered', () => {
const { sql, parameters } = compileFilters(NO_FILTERS);
// Not "no WHERE at all" as originally assumed: Kysely 0.28's `eb.and([])`
// compiles an empty conjunction to the truism `where 1 = 1` rather than
// omitting the clause (see parseFilterList in
// kysely/dist/cjs/parser/binary-operation-parser.js). That still matches
// every row, so it is the same "no filter" behaviour the old
// `clauses.length ? ... : ''` gave — the literal SQL text just differs
// from what was assumed here, which is what this assertion now checks.
expect(sql).toContain('where 1 = 1');
expect(parameters).toEqual([]);
});
it('matches a category and all of its descendants', () => {
const built = buildItemFilterSql(parseItemFilters({ category: '4' }), 1, null);
expect(built.clauses.join(' ')).toContain('RECURSIVE');
// One array parameter rather than one id: the CTE is seeded with ANY so
// several selected roots are walked in the same recursion.
expect(built.params).toEqual([[4]]);
const { sql, parameters } = compileFilters({ ...NO_FILTERS, categoryIds: [4] });
expect(sql).toContain('WITH RECURSIVE subtree');
expect(parameters).toEqual([[4]]);
});
it('seeds the descendant walk with every selected category', () => {
const built = buildItemFilterSql(parseItemFilters({ category: '4,9' }), 1, null);
const sql = built.clauses.join(' ');
expect(sql).toContain('RECURSIVE');
// ANY over the seeds is what makes several categories combine as OR: the
// result is the union of their subtrees.
expect(sql).toContain('= ANY($1::int[])');
expect(built.params).toEqual([[4, 9]]);
// One bind parameter holding the whole array, not a placeholder list. This is
// the property that made the array trap in the previous builder impossible
// here — see #297 and src/db-kysely/CONVENTIONS.md.
it('seeds the descendant walk with every selected category, as one parameter', () => {
const { parameters } = compileFilters({ ...NO_FILTERS, categoryIds: [4, 9] });
expect(parameters).toEqual([[4, 9]]);
});
it('requires every listed tag rather than any of them', () => {
const built = buildItemFilterSql(parseItemFilters({ tags: '1,2' }), 1, null);
// The count of matched tag rows must equal the number of tags requested —
// an ANY/IN match alone would return items carrying just one of them.
expect(built.clauses.join(' ')).toContain('COUNT(*)');
expect(built.params).toEqual([[1, 2], 2]);
const { sql, parameters } = compileFilters({ ...NO_FILTERS, tagIds: [2, 5] });
expect(sql).toContain('SELECT COUNT(*) FROM item_tags');
expect(parameters).toEqual([[2, 5], 2]);
});
it('numbers placeholders from the given starting index', () => {
const built = buildItemFilterSql(parseItemFilters({ category: '4' }), 3, null);
expect(built.clauses.join(' ')).toContain('$3');
it('filters on a price range', () => {
const { parameters } = compileFilters({
...NO_FILTERS,
minPriceCents: 1000,
maxPriceCents: 5000
});
expect(parameters).toEqual([1000, 5000]);
});
it('filters on status', () => {
const built = buildItemFilterSql(parseItemFilters({ status: 'reserved' }), 1, null);
expect(built.clauses.join(' ')).toContain('i.status = ANY');
expect(built.params).toEqual([['reserved']]);
});
// One clause for one status and for several, which is the whole reason the
// filter was generalised rather than joined by a second dimension.
it('filters on several statuses with the same single clause', () => {
const built = buildItemFilterSql(parseItemFilters({ status: 'available,reserved' }), 1, null);
expect(built.clauses).toHaveLength(1);
expect(built.clauses[0]).toContain('i.status = ANY');
expect(built.params).toEqual([['available', 'reserved']]);
it('filters on several statuses with one expression', () => {
const { sql, parameters } = compileFilters({
...NO_FILTERS,
status: ['available', 'reserved']
});
expect(sql).toContain('"i"."status" in');
expect(parameters).toEqual(['available', 'reserved']);
});
it('restricts to the favorites of the given customer', () => {
const built = buildItemFilterSql(parseItemFilters({ favorites: '1' }), 1, 42);
expect(built.clauses.join(' ')).toContain('EXISTS');
expect(built.clauses.join(' ')).toContain('favorites f');
expect(built.params).toEqual([42]);
const { sql, parameters } = compileFilters({ ...NO_FILTERS, favoritesOnly: true }, 7);
expect(sql).toContain('exists');
expect(parameters).toEqual([7]);
});
it('does not restrict to favorites when the flag is off, even given a customer', () => {
const built = buildItemFilterSql(parseItemFilters({}), 1, 42);
expect(built.clauses).toEqual([]);
expect(built.params).toEqual([]);
const { sql, parameters } = compileFilters(NO_FILTERS, 7);
expect(sql).not.toContain('exists');
expect(parameters).toEqual([]);
});
// Both routes reject this before reaching the builder, so it can only happen
// through a new caller that forgot to. Failing loudly beats dropping the
// clause and returning the whole catalogue as if it were someone's favorites.
it('throws rather than ignore a favorites filter with no customer', () => {
expect(() => buildItemFilterSql(parseItemFilters({ favorites: '1' }), 1, null)).toThrow();
});
it('continues numbering across multiple filters', () => {
const built = buildItemFilterSql(
parseItemFilters({ category: '4', min_price: '100', max_price: '900' }),
1,
null
expect(() => compileFilters({ ...NO_FILTERS, favoritesOnly: true }, null)).toThrow(
/favorites filter requires a customer id/
);
expect(built.params).toEqual([[4], 100, 900]);
const sql = built.clauses.join(' ');
expect(sql).toContain('$1');
expect(sql).toContain('$2');
expect(sql).toContain('$3');
});
});
// Both callers splice these clauses straight into query text, so a value
// reaching the clause string is SQL injection rather than a style problem. The
// comment on buildItemFilterSql says so; these two make it fail a build instead
// of relying on someone reading it. See #202, and #180 for the S2077 review.
describe('buildItemFilterSql keeps every value out of the SQL text', () => {
// Deliberately built by hand rather than through parseItemFilters, because
// the claim is that the clause literals are safe with no parser at all. These
// values could never survive parsing, which is the point: the parser is
// defence in depth, not the reason this holds.
const HOSTILE = "1); DROP TABLE items; --";
const hostileFilters = {
categoryIds: [HOSTILE],
tagIds: [HOSTILE],
minPriceCents: HOSTILE,
maxPriceCents: HOSTILE,
status: [HOSTILE],
favoritesOnly: true
} as unknown as Parameters<typeof buildItemFilterSql>[0];
it('never lets a filter value reach a clause, even one the parser would reject', () => {
const built = buildItemFilterSql(hostileFilters, 1, HOSTILE as unknown as number);
const sql = built.clauses.join(' AND ');
expect(sql).not.toContain(HOSTILE);
expect(sql).not.toContain('DROP TABLE');
// Every value still arrives, bound, where it can do nothing.
expect(built.params).toContain(HOSTILE);
});
// The structural version of the same claim, and the one that catches a value
// which happens not to look hostile: the SQL text must not depend on the
// values at all. Two disjoint sets of inputs, byte-identical clauses.
it('produces byte-identical SQL for two completely different filter sets', () => {
const a = buildItemFilterSql(
parseItemFilters({ category: '4', tags: '7,8', min_price: '100', max_price: '900', status: 'sold' }),
1,
42
);
const b = buildItemFilterSql(
parseItemFilters({ category: '99', tags: '11,12', min_price: '5', max_price: '6', status: 'available' }),
1,
it('composes several filters together', () => {
const { parameters } = compileFilters(
{
categoryIds: [4],
tagIds: [2],
minPriceCents: 1000,
maxPriceCents: null,
status: ['available'],
favoritesOnly: true
},
7
);
expect(parameters).toEqual([[4], [2], 1, 1000, 'available', 7]);
});
});
expect(a.clauses).toEqual(b.clauses);
expect(a.params).not.toEqual(b.params);
// The invariant, and it is load-bearing: the storefront call site is reachable
// without signing in, so a filter value reaching the SQL text is SQL injection
// rather than a style problem. These two made that fail a build rather than
// relying on someone reading a comment, and they still do — but they now check
// the SQL Kysely actually emits rather than the strings the old builder
// returned. See #202, #180 for the S2077 review, and #308 for the conversion.
describe('itemFilterExpressions keeps every value out of the SQL text', () => {
// Built by hand rather than through parseItemFilters, because the claim is
// that the expressions are safe with no parser at all. These values could
// never survive parsing, which is the point: the parser is defence in depth,
// not the reason this holds.
const HOSTILE = "1); DROP TABLE items; --";
it('never lets a filter value reach the SQL, even one the parser would reject', () => {
const { sql, parameters } = compileFilters(
{
categoryIds: [HOSTILE],
tagIds: [HOSTILE],
minPriceCents: HOSTILE,
maxPriceCents: HOSTILE,
status: [HOSTILE],
favoritesOnly: true
} as unknown as ItemFilters,
HOSTILE as unknown as number
);
expect(sql).not.toContain('DROP TABLE');
expect(JSON.stringify(parameters)).toContain('DROP TABLE');
});
it('produces byte-identical SQL for two completely different filter sets', () => {
const first = compileFilters(
{
categoryIds: [1],
tagIds: [2],
minPriceCents: 3,
maxPriceCents: 4,
status: ['available'],
favoritesOnly: true
},
5
);
const second = compileFilters(
{
categoryIds: [99],
tagIds: [98],
minPriceCents: 97,
maxPriceCents: 96,
status: ['sold'],
favoritesOnly: true
},
95
);
expect(first.sql).toBe(second.sql);
expect(first.parameters).not.toEqual(second.parameters);
});
});