feedback_ground_state_before_action Fired and Failed Five Times in One Day

The same correction appeared three times in one week: “wait, we thought we already committed that.” Ground state before action is a rule in our memory file, filed under feedback_ground_state_before_action.md. It has been there since April. The agent reads it on every session open, and it still fails on it roughly once every three days. We caught each lapse individually, treated it as a one-off, and moved on.

Our mental inventory of where Claude sessions went wrong was a collection of vibes: something about premature close, something about scope conflation, something about manifest drift. None was sorted by frequency, none was confirmed to be in the memory file yet, and none could be told apart from a one-session fluke. Corrections we made on Monday sometimes became new memory file rules and sometimes vanished back into the transcript. We couldn’t tell which without scrolling through the session history, and after the third time doing that in a week, we stopped. The corrective loop was running entirely in our heads, and it was lossy.

So we built a weekly retro script.

The audit that runs whether we remember to or not

The retro script lives in our ops repo. Every Sunday at 3 AM, a Windows scheduled task calls it. The script walks every transcript file under C:/Users/<username>/.claude/projects/, greps for correction and frustration markers, and clusters the hits by session and day. The markers are phrases we actually type when a session goes wrong: “wait,” “no,” “that’s wrong,” “we thought we,” “hold on.” Plenty of hits are routine disagreement, so the script reads the surrounding turns to separate those from actual redirects. Then it cross-references each cluster against the memory file. Any rule that matches a correction cluster gets flagged as fired-but-failed: the rule exists, the agent read it, and the session still produced the correction. Rules with no matching clusters get marked fired-ok, and rules that never appeared in any session this week get marked not-fired, which is its own signal.

The scheduling decision matters more than the script does. We had tried “we’ll review our AI mistakes at the end of the week” twice before. Both attempts died before week three. When the sprint fills, the optional review is the first thing to go. Sunday at 3 AM has no opinions about the sprint, and we can read the output over coffee without having to remember to generate it.

Stable section names are what make a diff possible

A flat list of complaints gets overwritten in memory by whatever happened in the most recent session. The design that works uses stable section names and consistent rule slugs, with the output landing at vault/04_Sessions/_retro/, where each weekly run diffs against the prior one. If feedback_ground_state_before_action appeared as fired-but-failed in week 3 and week 4, both rows show up. If a rule had no hits in week 5, the blank row is visible instead of missing. The trend is the evidence.

Our memory file has roughly 80 rules at this point. Most are feedback_*.md files with slugs like feedback_no_close_before_work_done or feedback_swarm_scan_proper. The retro reads each slug, finds whether any transcript hit matches it semantically, and marks it fired-ok, fired-failed, or not-fired. Duplicate rules surface as overlapping clusters. Rules that haven’t fired in three months appear with no hits in the report. That sometimes means the agent internalized them and sometimes means we stopped doing the work that would trigger them. You can’t tell which without the trend line, and the trend line only exists if the section names stay identical run to run.

Getting the output shape stable enough to diff was the harder part of the build. The grepping took an afternoon.

What the first full run found

The first seven-day run came back with twelve rule clusters and one finding we had not been able to name before seeing it. Session-footer cross-contamination ranked first by hit count.

The footer is the auto-generated block at the end of a session manifest: session ID, area, epic, task, current status. Ours was leaking context sideways. A session opened for one ticket would pick up stale content from a prior session’s footer, because the manifest builder was appending instead of replacing. The agent would start confident it was working on the previous ticket, because that was the last thing stamped in the footer layer, even when the open session had nothing to do with it. Opening exchanges felt off-track for days. We kept attributing it to sloppy session openers and never landed on a concrete cause.

The retro found it because “we thought we” appeared 7 times across 4 sessions in one day, all in the first three turns of each session. The rule feedback_ground_state_before_action was flagged as fired-but-failed on 5 of those. But that rule addresses the agent skipping a state check before acting. These sessions had a different failure mode: the agent started with wrong state already baked into the context, a footer from a prior session stamping the wrong ticket ID into the new session’s ground truth. The rule was right to fire, but the correction it pointed toward was wrong. Fixing the rule further would have changed nothing. The bug was in the footer writer, one layer below where the rule lives.

We patched the manifest builder the same morning: two lines that wipe the footer section before rewriting it instead of appending to it. The sessions that opened the next day were clean.

What stays invisible without a diff

Most AI workflow frustrations live as vibes because vibes don’t require evidence to sustain themselves. “Claude keeps doing this thing” is enough to feel annoyed. It gives you nothing to act on systematically, and nothing to check the last fix against.

The same decay shows up in infrastructure. The 80-rule memory file looks healthy. You added 12 new rules this month. You can’t see that the agent ignored 9 of the oldest ones without a run that compares this week against last. The ignored rules sit in the fired-but-failed column, waiting for a structured scan to make them visible as a pattern instead of noise.

The retro also distinguishes between a rule failing occasionally and a rule failing in a specific context. An occasional fired-but-failed hit might be a hard session. Five hits in one day, all in opening turns, all against the same rule, is a signal with a direction. That direction is only visible when you can compare it against prior weeks where the same rule was performing fine in those same positions. The footer contamination didn’t need a smarter rule or a more attentive team. It needed feedback_ground_state_before_action to appear in the fired-but-failed column specifically in opening-turn corrections, two runs in a row, which is how a pattern that looked like noise became a two-line fix on a Sunday morning.

Keeping the check running

Corrections that live only in someone’s memory get lost, and a fix can’t be shown to have worked without a comparison to the week before. Put a scheduled job in place that checks session transcripts against your written rules each week, and keep its section names stable so each run can be compared with the last. For any AI workflow, you should be able to see which rules the agent ignored and whether that changed after the last fix.

dxdev LESSON · paste into your AI coding agent
LESSON: A recurring correction can mean a bug one layer below the rule that keeps firing on it, and only a week-over-week diff reveals which
SOURCE: dxdev.com/blog/weekly-audit-ai-mistakes
WHAT HAPPENED: A memory rule requiring ground-state verification before acting had been lapsing roughly once every three days for months, with each instance individually corrected and then forgotten rather than tracked as a pattern. A weekly scheduled audit was built to grep correction and frustration markers out of session transcripts, cluster them, and cross-reference each cluster against the matching memory rule, diffing the result against the prior week's run. The first run surfaced a cluster of the same correction phrase appearing seven times across four sessions in one day, concentrated specifically in each session's opening turns. The existing rule was marked as firing but failing on most of those, but the real cause was one layer below it: a session-manifest footer was being appended to instead of rewritten, leaking a prior session's ticket id into a new session's starting context. Fixing the footer writer, two lines, resolved every following session.
THE RULE: A correction pattern repeating across sessions is not itself evidence that the matching rule needs to be restated or strengthened; it can be evidence of a mechanical bug one layer beneath the rule, and only comparing structured, dated clusters week over week, not memory of a recurring annoyance, reveals which one it is.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
      1. Any standing rule or instruction whose compliance is assumed from having been written down, rather than measured against actual transcript or log evidence over time.
      2. Any recurring correction where the applied fix has been to restate or strengthen the same rule more than once without checking for a mechanical cause underneath it.
      3. Any state-carrying file (a manifest, a footer, a config) that is built by appending new content rather than replacing it, which can leak a stale value from a prior run into a new one.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.