51 of 55. That is how many of the “already cleared” verdicts an adversarial audit refuted after we fanned a backlog review out to sub-sessions and got back 45 tidy files, each one confident.

Forty-five files that looked finished

A ticket migration had dumped 644 untouched comments into 45 “Story” buckets under one epic, 54 open items in all (45 Stories plus 9 real tickets). The job was to work out which comments described work that had shipped, died, or been superseded, so a human only had to decide about the survivors.

That is a fan-out shape. The Stories are independent, and each is a bounded corpus. A sub-session per Story can read git, read the live product, and write a verdict file. We dispatched them and got 45 files under a verdicts/ directory. Every file sorted its comments into SHIPPED, DEAD, SUPERSEDED, or still open. They read like finished work.

We started acting on them. Cleared items were collapsed out of our way, and the plan was to retire Stories on the strength of the files.

What the audit found

We ran a separate adversarial pass whose only job was to refute the cleared verdicts. The failure it found was systematic, and afterward we could name three causes.

“No commits on the ticket” was read as “never fixed.” Work in this codebase landed under weekly catchall tickets that never referenced the ticket that originated the request. A sub-session that greps for the ticket key finds nothing and concludes the work was abandoned. Another session that greps for the symptom finds it shipped. Each one is confident, and they disagree.

Story titles were a prior session’s guesses, and the workers adopted them. One Story was titled “never-built.” When we looked at it ourselves, the Dashboard had been built in February 2024 and shelved a day later. A worker told the frame is “never built” goes looking for confirmation of that frame. It found it.

The seed material was itself wrong. The migration’s own summary comment listed three “source-verified findings.” Two of the three were false. Every worker treated that comment as ground truth, so the error was copied 45 times instead of caught once.

The same pass held one more surprise. We had believed two tickets carried unmerged code, and it was four. Any retirement built on the SHIPPED/DEAD calls would have closed live work. The retire script now refuses those four by key, and we told the next session not to work around it.

What the wrong turn cost

The 45 files were unusable for the one thing we built them for. In the handoff to the next session we had to write, in capitals, that they were good for fragment and merge arithmetic (which comment is a duplicate of which). Every SHIPPED, DEAD, or SUPERSEDED call inside them was UNRESOLVED. We generated a confident-looking artifact, then spent more time teaching the next session to distrust it.

The replacement was slower and worked. We picked one coherent Story and did the clearing pass ourselves. We dated each item against known reworks, checked the live product, and checked git by symptom. Decisions went first, with cleared items collapsed at the bottom so they could be audited instead of read. One Story went from 22 comments to 6 decisions to one message from us, resulting in one carved-out ticket and a retirement. Three Stories retired that way, each walked item by item.

The same fan-out, working

Around the same time, we used four parallel reviewers on a site-templates ticket before release, and it held up. What came after the reviewers reported made the difference. A separate agent was tasked to adversarially verify the proposed fixes, and it found a real hole in one of them.

The fix turned on createBackup: 1 for the bulk-apply path. That made the bulk path the first caller ever to run Backup_WebsiteTemplate concurrently. Its ID lookup was only ever safe for one request at a time:

var newTemplateRow = SQL.top({
    select: "MAX(layouttemplateID) AS maxID"
  , from: "app." + prefix + "layouttemplate"
  , where: "username = '" + CleanSQL(username) + "'"   // username ONLY
});
if(newTemplateRow && newTemplateRow.maxID) {
  this.Add_WebsiteTemplate_Files(newTemplateRow.maxID, ...);
}

Bulk apply posts N requests in parallel, all inserting backup rows under the same username, and then each reads the global max. One request can pick up another’s ID, and that ID becomes the folder name. Site A’s files land in site B’s backup, and some backups get no files at all. The backup was the safety net for a path that deletes customer logos with no other copy.

The code itself showed this was real. The sibling lookup in Apply_WebsiteTemplate had already been hardened, with a comment saying it is scoped to the page so concurrent bulk requests cannot cross-assign each other’s inserted IDs. The backup path never got the same treatment, because before this change it was single-request only. The verifier also flagged its own limit: it had read the race, not executed it. That caveat is why we trusted it more than the 45 files, which carried none.

What separates the two runs

Both used parallel agents, and both produced confident output. The reviewer fan-out found a cache-poisoning bug and several file-handling bugs, and we trust it. The comment-triage fan-out produced 45 files we now trust for arithmetic only.

Three things differed.

  1. The review’s input was a diff. The triage input was 644 comments plus a summary someone had already framed. Workers cannot outrun a wrong frame they were handed.
  2. The review’s claims were checkable in one file. A race condition is a line of code. “This was never fixed” is a negative, and proving a negative needs a search strategy the worker may not have had.
  3. The review had a skeptic wired in. Someone was assigned to break the claims before they reached us. The triage run put nobody between the workers and us. We were the skeptic by default, which is how we paid for the whole run and could retire nothing from it.

Our rule now: any verdict of the form “this is done, dead, or superseded” gets audited against the symptom, and never the ticket key, before it moves anything. The word “cleared” has to be earned by something whose only job is to disagree with it.