The Fix That Looked Bigger Than It Was

Forty-three windows, closed by hand

Forty-three: that was the number that kicked off the post-mortem. One day, I’d closed forty-three agent windows manually, one at a time, no batch action, no shortcut. When I asked a session to reconstruct what had actually happened that day, it counted something else first: about half of everything I’d typed into any session that day was some version of “where are you.” Status checks. Not instructions, not corrections. Just me asking sessions where they stood.

The first read on that pattern is trust. I don’t trust the agents to tell me when something’s done, so I check constantly, and the checking itself generates the mess I then have to clean up by hand. It’s a plausible story. It’s also the wrong layer to fix, and the post-mortem only found that out by refusing to stop at the plausible story and going to pull the actual instrumentation instead.

What “delivered” actually meant

The mechanism under those forty-three windows was a delivery step that placed a window onto my screen and logged it as done. Placed, not seen. If I wasn’t at the keyboard when a session finished, the window still went up, still counted as a successful handoff, and then sat there in an empty room until I came back, noticed it, and closed it. Every one of those was a false positive: the metric said “delivered,” the ground truth was “nobody was there.”

That’s the same shape as a monitor that looks clean because it’s quietly excluding the thing that would make it dirty. The fix wasn’t “make the agents better at knowing they’re done.” It was presence detection: sessions now check whether I’m actually at the keyboard, and if I’m not, they hand back a link instead of placing a window nobody will see. The “success” event only fires when there’s someone there to receive it.

The watcher that had been lying for two days

Digging into the attention data to size the problem turned up a second, uglier finding. The watcher responsible for tracking whether I was present at the keyboard had been running stale code for two days. Not down, not erroring, just silently executing an old build the whole time. Which meant any conclusion drawn from “what attention data showed for the last two days” needed to be thrown out before it could be used as a baseline for anything, including the post-mortem that was currently trying to use it.

Same day, same audit, a third instrument turned out to be estimating instead of measuring: the trust ladder, the thing meant to track how often I approved a session’s output, was guessing at roughly 21 percent of my approvals rather than counting them. A number that looked like ground truth was partly invented.

None of these three were the same bug. But they were the same failure mode: a system reporting a clean signal that wasn’t actually derived from the event it claimed to represent. Fix the top-line symptom (I close a lot of windows by hand, so add more automation to close windows) and you’d have shipped something that made the dashboard look better without touching why the windows were empty in the first place.

The fixes that actually shipped were boring by comparison: presence-aware delivery instead of blind window placement, the desk queue stopped hoarding requests from sessions that had already closed (another source of phantom backlog inflating the “still pending” count), the watcher got put back on live code, and the trust ladder started counting real approvals instead of estimating them. A full handoff went onto the ticket for whoever picked up the thread next.

The pattern showed up again the same day

A separate ticket filed that same session was for an order-dependent test hiding inside the suite’s own baseline: a test whose pass or fail outcome depended on what ran before it, quietly living inside what the team treated as “green.” Different subsystem, identical shape. The baseline you’re diffing your fix against has to be verified as a baseline before the diff means anything. A metric that improved because the thing measuring it changed underneath you isn’t a fix. It’s noise wearing a fix’s clothes, and the only way to tell the difference is to go open the instrument itself before you believe what it’s reporting.

dxdev LESSON · paste into your AI coding agent
LESSON: A delivery metric that fires on placed rather than seen will report success on a handoff nobody actually received
SOURCE: dxdev.com/blog/2026-08-28_verify-fix-metrics-baseline-shift
WHAT HAPPENED: A pattern of forty-three manually closed agent windows in one day, alongside roughly half of all typed input that day being pure status-check questions asking where a session stood, was first read as a trust problem, not trusting agents to report their own completion, prompting constant manual checking. The actual mechanism was a delivery step that counted a session as successfully delivered the moment it placed a window on screen, regardless of whether anyone was actually at the keyboard to see it, so every session finishing while the operator was away still logged as a success and sat unseen until manually noticed and closed. The same investigation, digging into presence data to size the problem, found the presence-detection watcher had silently been running two-day-old stale code the entire time, not crashed and not erroring, just executing an outdated build, meaning any attention data from that window was unusable as a baseline until it was thrown out. A third instrument, meant to count real approvals, was found to be estimating roughly a fifth of its numbers rather than counting them, making a figure that looked like ground truth partly invented. A separate, same-day ticket found an order-dependent test hiding inside what the team treated as a clean baseline, the identical shape of bug, an instrument reporting a signal not actually derived from the event it claims to measure, recurring in a completely different subsystem.
THE RULE: Before trusting that a fix improved a metric, verify the instrument producing that metric was not itself stale, broken, or estimating during the measurement window; a metric reporting a clean signal that is not actually derived from the real event it claims to represent will make an unrelated fix look like it solved the problem when the real defect was in what was being measured, not in the top-line symptom.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
      1. Any success or delivered event that fires when an action is taken, such as placing a window or sending a notification, rather than when its intended recipient actually receives or acts on it.
      2. Any metric, dashboard, or baseline being used to evaluate a fix's impact without first confirming the instrument producing that metric was running current code and actually measuring, rather than estimating, the event in question.
      3. A recurring manual-cleanup pattern, such as closing windows or clearing a queue, addressed by adding more automation to do the cleanup, without first checking why the items needing cleanup were generated in the first place.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.