---
title: "The Feedback Loop That Hunts Its Own Bugs"
canonical: https://dxdev.com/blog/2026-07-13_self-improvement-feedback-loop/
datePublished: 2026-07-13
---
Our agent systems were producing more edge cases than I could reasonably hunt for by hand. The evidence was scattered across transcripts, investigation files, task history, and commits.

One isolated failure might be a prompt oddity. The same failure pattern appearing across runs is different. It is a candidate for a durable fix.

So I built a loop around that distinction. Every night, a scheduled job mines the record of agent work for lessons. When it finds a recurring failure pattern, it files an investigation instead of leaving a loose observation in a transcript. The investigation can query indexed git history, which matters because it can ask a question a transcript alone cannot answer: is this behavior still present, or did we already fix it in a later commit?

At 12:06 AM, the first investigation the loop generated closed, and it was a scanner false positive. That was the result I wanted from night one. Not because I wanted a bad signal, but because false positives are where an automated feedback system proves whether it is actually useful. A system that only describes its own successes is a reporting layer. A system that can catch a bad read, open the work, correct the scanner, verify the correction, and close the investigation has started to become operational.

The results flow into a self-improvement console, but the console is not the product. It is the evidence trail. The actual product is the loop from recurring pattern to investigation, diagnosis, change, verification, and closure.

## I did not want a better pile of notes

The sequence is intentionally simple:

```text
nightly schedule
  -> mine agent transcripts for recurring failure patterns
  -> file an investigation
  -> query indexed git history during diagnosis
  -> make and verify a correction
  -> record the lesson in the self-improvement console
```

There were easier versions of this system. I could have generated a daily summary of odd agent behavior. I could have put every suspicious event into a dashboard. I could have treated the transcripts as an inbox for manual review.

All three options lose at the same point. They preserve information, but they do not create a unit of work with a path to resolution. A summary is easy to skim and easy to forget. A dashboard lets a problem wait in public. Manual pattern hunting means the system only improves when somebody remembers to go looking for the same shape of failure across separate runs.

The other bad option was to let the agents auto-repair every anomaly they found. The first night showed exactly why I did not want that. A finding is not a bug merely because a scanner can name it. The loop needs an investigation stage between detection and change, and it needs verification after the change. That is not bureaucracy. It is the control that keeps a noisy detector from turning into a noisy repair bot.

I also kept the git index inside the loop for the same reason. The transcripts can show that agents repeatedly encountered a condition. They cannot, by themselves, establish whether the repository now contains the answer. Searchable history gives the investigation a way to distinguish an unresolved pattern from an already superseded one. Without that step, the system would keep reopening dead work and calling it improvement.

## The first finding was not the bug it appeared to be

The initial investigation did not uncover a new product defect. It uncovered a scanner false positive.

That is the diagnostic path I care about preserving. The scanner produced a signal that looked like a recurring failure worth investigation. The investigation did its job and found that the signal itself was wrong: every flagged hit was a synthetic plumbing-test prompt, one of my own automated test seeds, not a real correction the system had needed to make. We added a content blocklist to the scanner, and a re-scan confirmed it: the flagged pattern went from three hits in the prior week to zero.

The build itself took five minutes, from 12:01 to 12:06 AM. The work log also records a separate follow-up filed the same night on flagged spend paths, unrelated to the false positive. That second detail matters because the loop did not turn one bad signal into a reason to distrust every signal. It separated the incorrect read from other work that still needed a tracked follow-up.

That is the behavior I am after. A feedback loop should not be fragile enough that one false alarm poisons the whole system. It should be able to say, with a record behind it: this finding was wrong, here is the component that was wrong, here is the fix, and here is the verification.

## The unit that matters is a closed loop

The tempting metric for agent systems is output volume. How many transcripts did we mine? How many lessons did we extract? How many tickets did we create?

Those are activity metrics. They do not tell me whether the system got better.

The useful unit is the closed loop: a recurring pattern becomes an investigation; the investigation is checked against the code and history; a change is made only when the diagnosis supports it; the result is verified; the record makes the next diagnosis better. In this case, the loop's first completed cycle improved the scanner that feeds it.

That is a narrow result, but it is real. We now have a nightly process that can turn accumulated agent behavior into investigated work, use repository history as part of the diagnosis, and leave behind a verifiable correction rather than a theory about what the agents might be doing wrong.

The re-scan is the whole test for whether a loop like this is real. Not that it found something. That it could prove, with a number, that the something was gone: three hits down to zero.