---
title: "A 42 second sessions endpoint and a status color we trusted too long"
canonical: https://dxdev.com/blog/rubric-verifier-autonomous-workers/
datePublished: 2026-08-21
---
## Thirteen cards, five real

The secretary dashboard was showing thirteen active-work cards, and only five were real sessions. The other eight were zombie manifests: leftover session files that never got cleaned up when a run ended, still rendering as if an agent were mid-task. We wrote a filter that took the board from 13 cards down to 5. It also closed a related duplicate-manifest bug where the same session was spawning two ghost entries.

That fixed the surface bug. The problem underneath was a different one, and the filter didn't touch it.

Nearly two-thirds of the board was noise we had been treating as real. We don't know how much time we've spent clicking into a stale manifest expecting a live session and finding nothing, but it happened often enough that a filter felt worth writing. We trusted the status column for longer than we should have. Only then did we ask whether the thing setting the color could be wrong in the first place.

## What "amber" actually meant

Cards on the dashboard have a status color. Amber means the agent is saying this needs your attention, come look. The dashboard was rebuilt that day into something closer to a real operating surface: live per-agent progress inside each fleet card, drill-in on demand, and spawn-from-task so a worker can be kicked off directly from a ticket instead of a terminal. All of that made the board more useful to look at. None of it changed what amber was reporting.

Amber was self-report. A worker finished a task, decided for itself that the task was done, and flipped its own card to amber so we'd come review it. There was no step between "the agent thinks it's done" and "it's now sitting in our queue demanding a decision." Every card asking for judgment was just an agent's opinion of its own output. On a day with 33 sessions and 107 commits across 14 repos, that opinion is the bottleneck. If half of what lands in amber turns out wrong on inspection, we're doing the verification the worker skipped, one card at a time.

We had been running this dashboard for weeks before we asked that question directly. Amber meant "come look," and we came and looked. We assumed a worker declaring itself done was at least a reasonable signal for where our attention belonged. We never had a real count of how often that signal was wrong until the zombie-manifest cleanup forced us to look at every card on the board at once. Once the fakes turned up, we stopped trusting the five that were left.

## The contract change

We'd already been running review cascades by hand elsewhere. The docs work on the main product that day went through one model's review pass, then a second, independent model's pass, before it shipped. The blog redesign went through the same pattern: one review round and then a second, independent round, before going live. Those worked. But they were bolted on after the fact, a separate step someone had to remember to run. The amber problem was the same failure in a different shape: verification treated as optional and added back only when someone thought of it.

The fix came from a paper on agent-loop design, out of one of the frontier-model labs. It argues that a spawned worker should carry two things from the start. One is a rubric defining what success looks like, specific enough to check against. The other is a verifier, a second pass independent of the worker's own claim, that confirms the rubric was actually met. The standard becomes "the agent's output was checked against a stated bar by something other than the agent asserting it," where before it was "the agent says it's done."

We rebuilt the spawn seed, the initial task definition a worker gets handed, to require both fields. No rubric, no spawn. We also added a footer hook that runs at session close: verify-before-amber. A session cannot flip its own card to amber. It can only request a verification pass, and the card goes amber once that pass returns a result, whatever the worker declares.

## Why we kept review out of the after-the-fact slot

The obvious alternative was to keep doing what we were doing: let workers report done, and run review cascades manually on the important stuff. We rejected it for the same reason the zombie manifest bug existed in the first place. Manual steps get skipped when volume goes up. Eight stale manifests sat on the board because nothing forced cleanup, so they just accumulated. An after-the-fact review step has the identical failure mode. It works until the day is busy enough that someone forgets to run it, and busy is when you need it most.

The other alternative was to make the worker verify itself harder, with more test cases and a self-check pass before reporting done. We didn't trust that either, for the same underlying reason a zombie manifest doesn't report itself as stale. A process checking its own state has no independent signal when that state is wrong. The verifier has to be a separate pass with its own read of the rubric, because the same agent grading its own homework twice is still one opinion.

## What changed on the board

The sessions endpoint response time dropped from 42 seconds to 11 milliseconds in the same rebuild. That came from a TTL cache plus a single-flight fix that was overdue regardless of the rubric work. It mattered here because a faster board is what makes verify-before-amber tolerable. If checking a worker's status is slow, a mandatory verification hop on top would have made the dashboard worse to use. Fixing both at once meant the extra step cost no visible latency.

The doctrine is written up as a standing note in our archive (2026-06-11_loop-design-applied) and not left as tribal knowledge in one commit message, so the next spawn seed anyone writes inherits the same contract by default.

The eight zombie cards are gone and the five real ones are still there. When one of them goes amber now, a check passed it first. The worker's own word is no longer what puts it in front of us.