--- title: "The Test That Passed in Fixtures, Failed in Production" canonical: https://dxdev.com/blog/2026-08-28_rls-fixture-schema-drift/ datePublished: 2026-08-27 --- The alert fired three times in a row: `check:rls` exiting 1 after every listed check printed pass. I stared at that combination for longer than I want to admit, because a passing checklist with a failing exit code shouldn't be possible. ## The false green We'd just shipped `plan_approvals` under migration `20260827021000_boc495_plan_approvals`, part of the card-on-file billing flow at `/plan`. The migration made the table append-only by revoking `UPDATE` and `DELETE` at the schema level, keeping only `SELECT` and `INSERT`. That was a deliberate call on our part for an approvals ledger: once a plan approval is recorded, nothing should be able to rewrite or erase it, not even a bug. `check:rls` is the CI script we use to verify row-level-security policy coverage against the live schema. It ran, every check it printed came back passing, and it still exited 1. Three separate CI runs hit this over about 50 minutes: - 12:56 PM triage: "check:rls script exited with a failure after all listed checks passed; the specific failing assertion isn't shown in this excerpt" - 1:11 PM triage: "cause unclear: check:rls exits 1 after all listed checks pass; failing assertion not shown in log" - 1:41 PM triage: "cause unclear: check:rls script exits 1 after all listed checks pass; failing assertion not shown in log excerpt" Same symptom, three times, because three separate autofix attempts each patched a slightly different piece of the same gap, and each one missed something the others didn't touch. ## What the fixture actually tested The `check:rls` fixtures were written against an idealized permission model: a table has RLS enabled, policies exist for the roles that touch it, and the test suite exercises `SELECT`, `INSERT`, `UPDATE`, `DELETE` against a fixture schema that grants all four at the Postgres role level and lets RLS policies decide who can do what within that. That model is correct for every other table in our schema. It was never correct for `plan_approvals`, because `plan_approvals` doesn't have RLS deciding whether `UPDATE` is allowed. It has no `UPDATE` grant to decide over. The privilege itself is gone. The fixture and the live schema had drifted apart in a way our tests couldn't see: the fixture said "this table supports four operations, verify policies for each," and the live migration said "this table supports two operations, full stop." `check:rls` kept asking Postgres whether an `UPDATE` policy existed and evaluating it against a role that, on the real database, never had `UPDATE` in the first place. Depending on which assertion ran, that showed up as either a silent no-op (nothing to check, so nothing fails) or a downstream exit code that didn't map to any of the printed check names, which is exactly the "all checks pass, exit 1" signature the alerts kept reporting. ## Chasing it across three fixes - **12:58 PM** (`ci-alert/autofix`, 80.5s): first pass. Identified that `plan_approvals` was append-only via that migration and adjusted the script accordingly. Committed locally. Didn't fully close it. - **1:14 PM** (commit `commit-832D`, 77.9s): second pass, same root cause restated: "the migration revoked UPDATE/DELETE on `plan_approvals` to make it append-only, but `scripts/...`" still assumed otherwise somewhere else in the check path. Committed locally. - **1:44 PM** (commit, 91.7s): third pass, same diagnosis again, different line of the script. Watching the same diagnosis get rediscovered three times told me the bug wasn't in any single line, it was in the assumption baked into more than one place: the fixture data used to seed the RLS test schema, and the assertion logic that expected an `UPDATE` policy entry to exist for every writable table. Fixing the fixture without fixing the assertion left the assertion tripping on a table it could no longer reason about. Fixing the assertion without updating the fixture left the fixture asserting privileges the live schema didn't grant. ## Why this gets past test suites generally The test fixtures encoded permissions as data: a set of expected grants per table, checked in as fixture rows. The live schema encoded permissions as a migration, applied once, out of band from anything the fixtures read. Nothing forced those two representations to agree. A migration can revoke a privilege on production without a single fixture file changing, and the test suite will keep testing the old, wider permission surface until someone notices the exit code doesn't match the printed output, which is a much weaker signal than a failing assertion with a name. We're not patching the fixture again. We're making the RLS check introspect the live grant set instead of asserting against a static fixture, so `check:rls` fails loud and specific, "plan_approvals: UPDATE not granted, skipping UPDATE policy check," instead of falling through to an unlabeled exit 1. A test that can't tell you which assertion failed is a test that will cost you three fix attempts before someone thinks to ask what privileges the table actually has.