The weekly total came back as 46.9M output tokens and 9.8 billion cache reads, and we read the two numbers twice because we assumed we had misplaced a decimal. We hadn’t. Over seven days, for every token an agent wrote, about 209 tokens were read back from cache.

Every agent session we run leaves a JSONL transcript, and each assistant turn carries a usage block with four counters: input_tokens, output_tokens, cache_creation_input_tokens and cache_read_input_tokens. We had never summed them. The daily work log already prints them per session, so we wrote a small script to walk the transcript folders and add up the same four fields for the week.

The wrong turn: we shrank the model

Before we looked, we had a theory. The bill felt too high, and the obvious lever was the model. So we acted on it. The scheduled jobs, the ones that fire claude -p every half hour to classify backlog items and produce the morning digest, all went down to Sonnet at medium effort. Our interactive work stayed on the bigger model, but we started tolerating weaker output there too, because cheaper felt like progress.

It cost us two things. The first was quality. The classification workers started disagreeing with us, and one afternoon we moved five tickets between boards on the strength of a classification rule that was wrong, then reversed three of them by hand. The rule was our error rather than the model’s, but we had gone looking for savings in exactly the place where a small mistake gets expensive. The second was that the change did nothing to the number we were worried about. When we finally did the arithmetic, the scheduled workers were the cheapest thing on the machine.

Look at one of those workers in the log. A single classification run: 1 prompt, about 20,000 output tokens, 51,766 cache-read tokens. Run that every 30 minutes and you get roughly 2.5M cache reads a day. Against the weekly total, that is a rounding error.

What was actually big

Here is one interactive session from the same day, straight from the log:

1 session, 397 prompts
in=1,376  out=425,998  cache_read=168,910,001

And the session that made us write this up: 483 turns, 128M cached tokens read against 428k of fresh input plus output combined. That is roughly 300 to 1 inside a single conversation.

The mechanism is plain once you say it out loud. Each turn re-sends the whole conversation so far. The provider serves most of that prefix from cache, which is why it is cheap per token, but “cheap per token” times “the entire history” times “every turn” is the whole bill. The cost of turn N is roughly proportional to everything said in turns 1 through N-1. Sum that over 483 turns and you get an area under a curve that grows with the square of the thread length. The fresh tokens we typed and the tokens the model wrote are a thin sliver on top.

Averaged out, that 483-turn session was reading about 265k tokens of context on every single turn. Most of it was old tool output: file dumps, grep results, log tails from things we had dealt with two hours earlier and never needed again.

Why the model swap couldn’t have worked

Cache reads are billed at a fraction of the normal input rate, and a cheaper model discounts that rate further. But a discount on a multiplier doesn’t change the multiplier. If a thread is 10 times longer than it needs to be, you pay for that on any model. Moving the scheduled jobs down a tier shaved a little off a line item that was already tiny, while the long interactive threads kept accruing quadratically.

The lever that matters is thread length times turn count. Halving the model price halves the bill. Cutting a 483-turn thread into four threads of roughly 120 turns each does far better than that, because the sum of average context sizes drops with the square, not linearly. Four short threads each carry a small history. One long thread drags its whole history through every turn.

What we changed

Nothing clever.

  1. Close threads at a natural seam. Ticket shipped and verified, thread closes. The work log entry becomes the handoff, and the transcript does not. That day’s log has 106 sessions in it, and the ones that finished cleanly are the cheap ones.
  2. Write state down, then start fresh. When a task has to survive a break, we write the resume state into the ticket (what is done, what is parked, what decision is pending) and open a new session that reads only that. A page of notes replaces a quarter-million tokens of scrollback.
  3. Stop dumping raw output into the thread. A tail of a log we need for one decision gets read once and summarized in a sentence. It doesn’t ride along for the next 300 turns.
  4. Put the model back. The interactive work is on the model we trust for it again. The scheduled workers stay on Sonnet because they are cheap and their job is narrow, which is a fine reason for them to be there, even though it was never a savings plan.

The number we watch now

We added the aggregation to the daily sync, so every session line now shows in / out / cache_read. The column we look at is cache_read divided by turns. When one session’s figure climbs past a couple hundred thousand per turn, that thread has outlived its usefulness, and the move is to write down where we are and close it.

The 9.8 billion figure doesn’t scare us anymore. It is mostly a measure of how long we let conversations run, and that is a habit we can change without touching a single model setting.