Files
redefined-designs/backend/tests/integration/setup/testDb.ts
T
synAdminandClaude Opus 5 2bc9440b38 test(ci): record which Postgres actually answered (#154)
This adds the measurement #154 has needed twice and never had. It does not fix the failure and does not guess at it: the issue has been wrong twice from reasoning ahead of evidence, and the point here is to make the next occurrence answer the question rather than reopen it.

The observation that rules out every explanation so far is that the schema comes back. A suite fails because orders does not exist, and a later suite truncating that same table passes. A dropped database does not un-drop itself, so this was never one database losing its schema. More than one server answering to one name produces exactly this, and Docker embedded DNS round-robins every container sharing an alias, so a leftover service container from an earlier run fits every observation including the empty dmesg that killed the OOM theory.

globalSetup now logs every address the database host resolves to. More than one is the answer outright. One address means this reading is wrong too, and the next suspect is a single container restarted with a fresh data directory.

Alongside it, both globalSetup and a failing assertSchemaPresent record which server actually answered. pg_postmaster_start_time is what settles that and needs no special rights: two Postgres instances cannot share one, so differing values within a single run are proof, where a differing inet_server_addr alone could be argued to be one container that moved. The failure message now says to compare the two rather than leaving the reader to know that is the interesting comparison.

Logged on a passing run as well as a failing one, deliberately. A failing run's addresses mean nothing without a passing run's to compare them against, and this issue has twice suffered from having only the failure to look at.

Neither can throw. A diagnostic that fails the run it was added to explain is worse than no diagnostic, so both are wrapped and both degrade to a printed reason.

The failure path costs one extra round trip, taken only when the schema is already known to be missing. assertSchemaPresent is not on the hot path — resetDb calls it only when its TRUNCATE has already failed.

Verified: tsc clean, typecheck:tests clean, lint 0 errors with no new warnings, and the four message patterns schemaLoss.integration.test.ts asserts on are all still present. The probe reads only pg_catalog functions, so it still answers against the dropped schema that suite creates.

Refs #154

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-09 10:49:03 -05:00

188 lines
7.5 KiB
TypeScript
Executable File

import { Pool } from 'pg';
import { runner } from 'node-pg-migrate';
import path from 'path';
export const testPool = new Pool({
host: process.env.TEST_PGHOST || 'localhost',
port: parseInt(process.env.TEST_PGPORT || '55432', 10),
user: process.env.TEST_PGUSER || 'redefined_test',
password: process.env.TEST_PGPASSWORD || 'redefined_test',
database: process.env.TEST_PGDATABASE || 'redefined_test'
});
export async function migrate(): Promise<void> {
await runner({
databaseUrl: {
host: process.env.TEST_PGHOST || 'localhost',
port: parseInt(process.env.TEST_PGPORT || '55432', 10),
user: process.env.TEST_PGUSER || 'redefined_test',
password: process.env.TEST_PGPASSWORD || 'redefined_test',
database: process.env.TEST_PGDATABASE || 'redefined_test'
},
dir: path.resolve(__dirname, '..', '..', '..', 'migrations'),
direction: 'up',
migrationsTable: 'pgmigrations',
count: Infinity
});
}
/**
* The tables `resetDb` truncates. Also, therefore, exactly the set whose absence
* makes this suite meaningless.
*/
const REQUIRED_TABLES = [
'admin_settings',
'cart_items',
'carts',
'categories',
'checkout_items',
'checkouts',
'customer_sessions',
'customer_tokens',
'customers',
'favorites',
'item_drafts',
'item_images',
'item_tags',
'items',
'orders',
'shipping_addresses',
'tags',
'upload_links'
] as const;
/** Which of them the database does not currently have. */
async function missingTables(): Promise<string[]> {
const { rows } = await testPool.query<{ table_name: string }>(
`SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public' AND table_type = 'BASE TABLE'`
);
const present = new Set(rows.map((row) => row.table_name));
return REQUIRED_TABLES.filter((name) => !present.has(name));
}
/**
* Says plainly when the schema has gone, rather than letting the suite report
* it as unrelated logic failures.
*
* In #154 the integration run lost its schema partway through — a Postgres
* service container recreated mid-run comes back with an empty data directory —
* and it presented as 36 assertion errors about categories, price filters and
* favourite notifications. The real message, `relation "items" does not exist`,
* was further down the same log. Hours went into chasing the assertions.
*
* The cause is still open and needs runner-side evidence. This is the half that
* can be fixed from here: whatever the cause, the next occurrence should read as
* "the database lost its schema" on the first line.
*/
/**
* Which Postgres actually answered, rather than which one we asked for (#154).
*
* The reason this exists: the schema in a failing run **comes back**. A suite
* fails because `orders` does not exist, and a later suite truncating the same
* table passes. A dropped database does not un-drop itself, so more than one
* server must be answering to the same name — Docker's embedded DNS round-robins
* every container sharing an alias, so a leftover service container from an
* earlier run would produce exactly this.
*
* `pg_postmaster_start_time()` is what settles it and needs no special rights:
* two Postgres instances cannot share one. Differing values across a single run
* are proof outright, where a differing `inet_server_addr` alone could be argued
* to be one container that moved.
*
* Never throws. This is a diagnostic, and a diagnostic that can fail a run it
* was added to explain is worse than no diagnostic.
*/
export async function describeBackend(label: string): Promise<string> {
try {
const { rows } = await testPool.query<{
addr: string | null;
started: string;
pid: number;
db: string;
}>(
`SELECT inet_server_addr()::text AS addr,
pg_postmaster_start_time()::text AS started,
pg_backend_pid() AS pid,
current_database() AS db`
);
const row = rows[0];
if (!row) return `${label}: no row returned`;
return `${label}: db=${row.db} addr=${row.addr ?? 'local'} postmaster_start=${row.started} pid=${row.pid}`;
} catch (err) {
return `${label}: could not be identified (${err instanceof Error ? err.message : String(err)})`;
}
}
export async function assertSchemaPresent(context: string): Promise<void> {
const missing = await missingTables();
if (missing.length === 0) return;
// Gathered only on the failure path, which is the one worth paying for, and
// is where #154 has repeatedly lacked the one fact that would identify it.
const backend = await describeBackend('Answering server');
throw new Error(
`The test database has no schema (${context}).
` +
`Missing ${missing.length} of ${REQUIRED_TABLES.length} tables: ${missing.join(', ')}.
` +
`${backend}
` +
`Migrations ran at the start of this run, so the schema existed and has since gone. ` +
`Nothing in this suite drops tables — TRUNCATE does not. Compare the postmaster start ` +
`time above against the one globalSetup logged: if they differ, this is a different ` +
`Postgres instance answering to the same name rather than one database losing its ` +
`schema, and the schema was never lost at all. ` +
`See #154.`
);
}
export async function resetDb(): Promise<void> {
// The schema check runs only when the truncate fails, not on every reset.
// resetDb runs in a beforeEach several hundred times a suite, and an extra
// round trip each time to guard against something that has happened once
// would be paying continuously for a rare event. A failure here is where the
// information is worth having.
try {
await testPool.query(`
TRUNCATE TABLE item_drafts, upload_links, orders, checkout_items, checkouts,
shipping_addresses, cart_items, carts, customer_tokens, customer_sessions, favorites,
customers, item_tags, item_images, items, tags, categories
RESTART IDENTITY CASCADE
`);
} catch (err) {
await assertSchemaPresent('while resetting between tests');
// The schema is there, so this is not #154 — let the original speak.
throw err;
}
// admin_settings is not truncated — it holds the seeded cart_expiry_hours
// default that other suites read. But the email template rows in it are test
// data like any other, and a stored template outliving the suite that wrote
// it silently changes the mail every later suite asserts on. That is not
// hypothetical: a subject of "Gone" written by the template tests reached the
// favorite-alert tests and made five of them fail somewhere else entirely.
//
// Same reasoning, same failure, for the intake settings (#227): a ceiling of
// 1 left behind by the ceiling suite makes every later submission refuse with
// a 503, in files that never mention a ceiling.
//
// ESCAPE, and not a backslash, because the backslash that used to be here did
// nothing. In a JavaScript string 'email\_%' is 'email_%', and `_` in SQL LIKE
// matches any single character — so the pattern meant "email plus any one
// character", not "email_". It happened to delete the right rows only because
// no other key begins with those letters followed by something else; a
// setting called emailing_enabled would have been swept away silently. Found
// by no-useless-escape the first time lint was pointed at tests/ (#298).
await testPool.query(
`DELETE FROM admin_settings WHERE key LIKE 'email!_%' ESCAPE '!' OR key LIKE 'intake!_%' ESCAPE '!'`
);
}
export async function closeDb(): Promise<void> {
await testPool.end();
}