The Expensive Part Was The Memory

For ten days we ran a team of seven AI agents on Accent, the content management system we build. They built, tested, reviewed and shipped fixes in batches while a person made the calls. Then the weekly meter showed a quarter of the allowance gone in one day, which puts the end of the week on day three.

The team

Accent was built one Claude Code session at a time: one task per session, every change approved by hand. That works, but the person approving is the bottleneck. We wanted the work to keep moving when nobody was watching it.

OpenRig, an open-source tool, runs a team of coding agents side by side, each in its own seat with its own role. Ours started as a builder and a reviewer. Within two days it was seven seats: a lead who writes the briefs, a builder, a tester who reruns everything on a clean copy, a designer, two reviewers and an advisor. Above them sat a coordinator we called Hermes. It approved each push, checked changes in a browser against a copy of the database, and shipped when told to.

Every fix took the same route: brief, build, test, review, a second review when it touched saving or stored data, a browser check, then a deploy that compares every page before and after. One batch closed fifteen issues.

The guess

The agents were set to the second-highest effort level, which means more thinking on every step. That looked like the obvious cost, and turning it down looked like the obvious fix. We measured first.

The measurement

Claude Code writes each session to a JSONL file, and every response carries a usage record: tokens written to the prompt cache, tokens read back from it, and tokens of output. Summing them across every session on the account since the weekly reset takes a few lines:

for line in open(path):
    entry = json.loads(line)
    if entry.get("type") != "assistant":
        continue
    usage = entry["message"]["usage"]
    totals["cache_read"] += usage.get("cache_read_input_tokens", 0)
    totals["cache_write"] += usage.get("cache_creation_input_tokens", 0)
    totals["output"] += usage.get("output_tokens", 0)

One detail: a streamed response writes one line per content block, all carrying the same usage, so count each message id once.

About a day and a half came to 4,381 steps, 2.26 billion tokens read back from the cache, 25.7 million written to it, and 5 million of output. Weighted the way the API prices them (a cache read at a tenth of an input token, a cache write at twice one, output at five times), about 75% of the cost was reading, 17% was writing, and output, which is where effort shows up, was under 10%.

What was expensive

A model has no memory between steps. Each step sends the whole conversation again, and the prompt cache makes the resend cheaper, not free. Our agents ran with a million-token context window and compacted only when it was nearly full, so a typical step re-sent between 450 and 600 thousand tokens. The work in a step was small. The history carried into it was not.

The largest single reader was not a builder. It was the coordinator, at about 27% of all cache reads. Its context had grown to nearly a million tokens, and it woke for every push to approve and every change in the queue, re-reading most of that each time. The seat that existed so a person could stay in control was the most expensive seat on the team.

What we changed

• The team stays as it was: seven seats, the same checks. Effort went up to the highest setting, since thinking was never the cost.
• The coordinator starts fresh for each batch from a short handover note, and wakes only for a push, a batch ready to ship, or something stuck.
• Next: each agent starts every task with a fresh context instead of carrying the last task’s history.

In a week the same script runs again, and the two weeks get compared step for step.

The general point

With agents, cost follows context length times the number of steps, not how hard each step thinks. Keep contexts short, wake agents only when there is work for them, and measure from the logs before cutting anything. The part you were sure about is often not the part.