The agent asked for a JIRA issue and got back { expand: string[82], fields: { ... } }. A few minutes later, rtk git log -1 --oneline showed a non-merge commit at the tip, though the merge had landed. We had assumed the token-saving filter was a safe default for every command. The status check and the merge check both depended on literal fields and a merge node that the filter could remove.
We run a CLI wrapper that compresses verbose command output before it reaches our AI agent. It groups thousands of lines of test failures into a summary, collapses a noisy git diff into the parts that changed, and turns a wall of JSON into a shape. We were burning through a Max plan running agents all day, and the compression takes 60 to 90 percent off the token bill on the commands we run constantly: git, curl, gh, test runners. It saves real money.
It also, on the one afternoon we most needed authoritative output, handed our agent a sketch of our data instead of the data, and hid the exact commit we were trying to confirm had landed.
Closing a ticket from the wrong machine
We run a legacy SaaS on Windows, IIS, and SQL Server, plus a growing pile of agent tooling on a desktop. That afternoon we tried something we had never done: close a ticket in the product repo without opening a session inside that repo. The code lived in one clone. Our agent was rooted in a different station entirely, reaching across to drive the work over the JIRA REST API and the git CLI from the outside.
The close itself is mechanical. Merge the feature branch into the integration branch, push, clean up the local branch, fire the JIRA transition that moves the ticket to its shipped state, verify in a browser. Nothing exotic.
Reaching across machines tested every assumption. Most of the friction that day was boring and Windows-shaped. There was no /tmp directory, so a hardcoded temp path blew up. Quoting got ugly trying to run Python inside a bash heredoc inside a Windows shell. A git branch -d refused to delete because the local branch carried a merge commit the remote did not have yet. Papercuts.
Two of the problems were more serious. In both, the tool we trusted told our agent something false.
Trap one: a schema sketch where the payload should be
Part of the close is verifying the JIRA transition took. So the agent pulled the issue back over the REST API to read the field values and confirm the new state. The fetch went through the wrapper, like every other command.
What came back was the line at the top of this post:
{ expand: string[82], fields: { ... } }That is a description of the issue, not the issue. The wrapper recognized a big JIRA REST payload and decided a human looking at it would not want 82 expand options and a hundred nested field objects dumped to the terminal. So it replaced the contents with their types and counts. For a person skimming, that helps. For an agent that has to read the literal value of a status field to confirm a transition, it is a dead end. The data the agent came to read was not in the output. Only its shape was.
The agent reported that it could not verify the field values. That was the right thing to report, but the transition had in fact succeeded. The verification failed because the tool in the middle had eaten the evidence, while the work itself had gone through.
Trap two: the merge commit that wasn’t there
Next we went to confirm the merge landed on HEAD. Standard move: rtk git log -1 --oneline, read the top commit, check it is the merge.
The output showed a non-merge commit. According to it, the merge had never landed.
It had. A minute later we confirmed it with git rev-parse HEAD and a raw git log with no wrapper in the path. The merge node was right there. The compactor, trying to give a clean one-line history, had collapsed or suppressed the merge node specifically and surfaced a different commit as the tip. The one piece of git state we were checking was the one piece the filter tidied away.
A merge commit is the most load-bearing object in a release. It is what you check before you tell anyone “it shipped.” The tool dropped it from view in the name of readability.
Why this was guaranteed
It would be comforting to call this a bug. It was the tool working as designed, and that is the uncomfortable part.
The wrapper is structural. It does not blindly truncate text. It recognizes what kind of output it is looking at, a git log, a REST response, a test report, and applies a format-aware transform to make it pleasant for a human to read. That intelligence is the whole value. But every one of those transforms assumes a human is going to read the result.
Agents parse. When a person reads { fields: { ... } }, they mentally fill in “the actual fields are in there somewhere.” When an agent parses that same string, the fields are gone. The token that carried the value was deleted and replaced with a token that carries its type, and nothing in the summary lets the agent recover the original. Lossy-but-pretty is fine for eyes. For a machine doing verification, lossy is just wrong.
The two failures were the same failure in different clothes. One collapsed a payload into its schema. The other collapsed a history into a cleaner but false tip. Both times the tool optimized for the reader and sacrificed the exact bytes the agent needed.
The fix and the rule it forced
The fixes were small and immediate. For the API call, we stopped shelling out to curl and made the request from inside Python with urllib. No external command means no wrapper in the path, and the agent gets the raw JSON back with every field intact. For the SHA, we used git rev-parse HEAD and raw git for anything where exact object identity matters, never the compacted log.
In practice the bypass is three lines you can lift:
# verify object identity: raw git, never the compactorgit rev-parse HEAD # exact SHAgit log -1 --pretty=%H%n%P # tip + parents (two parents = merge)# verify an API field: no curl, no wrapper in the pathimport json, urllib.requestreq = urllib.request.Request(url, headers={"Authorization": auth})fields = json.load(urllib.request.urlopen(req))["fields"]assert fields["status"]["name"] == "Done" # literal value, intactThe rule those fixes left us with:
Any tool that sits between your agent and ground truth has to be lossless on the paths that matter. Pretty-printing is for humans. For machine verification, give the agent the raw bytes.
We baked that into the toolkit we were building. The git helper module that wraps subprocess calls for our scripts carries an explicit note at the top: raw subprocess wrappers, no compression filtering, because scripts need authoritative output. The token-saving wrapper is banned from any code path where a script is checking state, whether a SHA, a transition, a payload, or anything else load-bearing. It stays on everywhere else, still saving the money, still summarizing the 4,000-line test run down to the three lines that failed.
We did not throw out the optimizer. It is right most of the time and pays for itself daily. We drew a line around the surfaces where exactness is the entire point and routed those through raw bytes with no clever layer in between.
What to check on your own setup
If an AI agent verifies state for you, find out which tools sit between it and the raw data, and whether any of them summarize output. In our case a filter replaced a payload with its shape and hid a merge commit, and the transition and the merge had both succeeded. Keep the compression on for routine work, and send every check of a SHA, a status or a payload through raw output.
Related
- Finding 366 Silent rtk Failures in Our AI Agent Sessions: surfacing the failure patterns that a lossy filter can hide across many sessions
- Reverse-Engineering Claude Code’s Session JSONL (So You Can Analyze Your Own Transcripts): the transcript format the filter acts on; understanding it is the prerequisite to knowing when filtering is safe
- The CAPTCHA lockout you trigger yourself: when a wrong API endpoint hammers you to a 403: another case of tooling that looks fine but produces a silently wrong outcome
- Encoding Hard-Won Corrections as Code Defaults Instead of Trusting Yourself to Remember: the right follow-up to a filter failure, encode the bypass rule instead of relying on memory
- Same SHAs in Two Repos: Why Commit-Count Metrics Lie in a Clone Workflow: another category of plausible-but-wrong output from an automated tool