Why Our Release Check Passed Minutes Before the Meeting

The scan we run before every release meeting came back clean minutes before the meeting: board and code in sync, no discrepancies, nothing to chase down. That verdict was about to go straight into the meeting.

What the scan said, and who doubted it

The report was tidy. Every ticket the release diff touched had a matching entry on the board, no leftover references, nothing missing. It was a real check and it had passed. The person about to run the meeting pushed back anyway, from memory. Last time, something in a different part of the codebase hadn’t been flagged properly, and they wanted it checked again before anyone trusted a clean read.

Why a clean read was possible

The pushback was right, and the scan’s own design showed why. Every check it ran started from the pending code changes and asked whether the board agreed with them. That direction catches a ticket wrongly marked or missing from the diff. It cannot catch a ticket the meeting would read out loud that the diff never contained, because there is nothing on the code side to compare against. A scan built around “does the board match this diff” has no way to ask whether a ticket on the board should have been touched by the diff and wasn’t.

So we rebuilt the check the other way. It starts from every ticket on the board and asks what the code history says about it. On a release the narrower scan had called clean minutes earlier, it surfaced seven real discrepancies. Some tickets carried a release label whose code had never reached the branch that ships. At least one ticket’s work lived in the codebase under a different label than the board’s own field claimed.

The new rule that was wrong

The rebuilt check went in as three new rules, and one of them was wrong. It flagged two tickets as anomalies because their code lived only in a separate pipeline the main release doesn’t touch, and it treated that as a mismatch. It isn’t one. That pipeline is supposed to carry the same release label, by design, and it ships on its own cadence a little later.

We shipped the rule anyway, and for a while the meeting summary carried two false anomalies. About an hour later that pipeline’s actual release ran, which gave us something real to test the rule against. The rule was demoted from an anomaly to a plain note in the same meeting summary.

Where it landed

The release meeting now reads from a check that looks in both directions, board to code and code to board. The seven real gaps were found the same morning they became visible. The rule that flagged correct data was fixed before it could do that every week. The only reason the gaps surfaced at all is that someone declined to accept a clean report, given what had gone wrong before.

A check that starts only from the code changes cannot see a ticket the changes never touched. Run it from both ends, and test any new rule against a real release before you trust what it flags.

dxdev LESSON · paste into your AI coding agent
LESSON: A Coherence Check Built to Read One Ship List Can't See a Ticket That List Never Contained
SOURCE: dxdev.com/blog/the-pre-release-scan-was-checking-only-one-direction
WHAT HAPPENED: A pre-release scan reported the board and the code in sync, and that "in sync" verdict was about to go straight into a release meeting unchallenged. It was wrong: every one of its checks started from the pending code changes and asked whether the board agreed with them, which structurally cannot catch a ticket the meeting would read out that the pending changes never touched at all. Rebuilding the check to instead start from every ticket on the board and ask what the code actually says surfaced seven real discrepancies the old, narrower scan had been calling clean. The very first version of that rebuilt check then produced its own false positive, flagging two tickets as anomalies that had, in fact, shipped correctly through a different, valid pipeline; that was caught and corrected about an hour later, once the real release for that pipeline had actually run and could be checked against.
THE RULE: A coherence or reconciliation check that only walks in one direction (from the change set to the source of truth) cannot see a discrepancy that never produced a change in the first place; build it to walk from the source of truth outward instead, and expect its first version to have its own false positives worth checking rather than trusting immediately.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
  1. Any reconciliation, drift, or coherence check that starts from a diff or change set and asks "does the source of truth agree," rather than starting from the source of truth and asking "does the diff account for this."
  2. Any "all clear" or "in sync" report that would be relied on in a meeting or decision without a spot check against the underlying data first.
  3. Any newly added anomaly-detection rule that hasn't yet been run against a real, known-good case to confirm it doesn't fire on correct data.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.