How to Cut Agent Token Spend Without Cutting Capability

Coding agents on long tasks burn tokens in a way most teams never look at. The model doesn’t spend them on writing code. It spends them rereading the same growing context on every turn. NVIDIA’s SoL-Pi project measured this on two-to-twelve-hour runs, then searched for harness changes that cut the waste.

The result is an open-source extension for Pi, the coding agent toolkit from Earendil Works. On their held-out benchmark, it used 45-49% fewer tokens than Pi and cost about a third less. It also kept only about 94% of Pi’s score.

Our read: the four mechanisms are useful, but the method matters more. NVIDIA fixed its capability floor before searching for savings, and every change had to clear it. That’s the discipline most teams skip when they “optimize” agent cost. Without it, you can’t tell savings from an agent that quietly does less.

Key Takeaways

In SoL-Pi’s measurements, cache reads made up about 99% of Pi’s token traffic on long tasks. Replayed context is the target, not output.

NVIDIA accepted a change only if every capability metric stayed within a tolerance set in advance. About 1 in 40 ideas survived.

Four changes survived: fused edit-and-check calls, compaction only when it pays, archived large tool output, and cheap log summaries with verified quotes.

The combined harness kept about 94% of Pi’s score, and solved fewer Terminal-Bench 4 tasks. Efficiency gains have a capability price you should measure.

Before changing anything, pull a token breakdown by category from your own agent traces, and write down your capability floor.

Where do agent tokens actually go on long runs?

Most of them are context the model has already seen. The SoL-Pi paper (Liu et al., NVIDIA, September 17, 2026) reports token traffic by category on EdgeBench, a set of long-horizon coding tasks. For the base Pi harness on GPT-5.6 Sol, cache reads were 2.13 billion of 2.15 billion tokens.

That’s what an agent loop does. Every turn sends the whole conversation back: instructions, file reads, test logs, earlier reasoning. Prompt caching makes each reread cheaper, but the reread still happens. A 20 KB test log read on turn 30 gets replayed on every turn after that.

Cache reads are billed at a discount, so their share of your bill is smaller than their share of tokens. Check your own provider’s prices before you size the prize. But the shape holds: on long runs, cost grows with context size times turn count. In token terms, output and fresh input are a rounding error by comparison.

This changes where you look. Teams usually reach for a cheaper model first. We covered that lever in right-sizing AI spend, and it’s real. SoL-Pi works a different lever: what the harness puts back into context, and how many turns it takes.

Token category (GPT-5.6 Sol, EdgeBench)	Pi	SoL-Pi
Cache read	2.13 B	1.06 B
Cache write	0.014 B	0.032 B
Total traffic	2.15 B	1.10 B (-49.0%)

Source: SoL-Pi paper, Table 1 and body text. Fresh input and output make up the small remainder of each total. Note that cache writes more than doubled. Compaction rewrites the cache, and that cost is real.

How did NVIDIA decide which changes were safe?

They set the rules before the search, then let the search run. Every capability metric got a tolerance, fixed before the experiments began. A candidate passed only if every capability metric stayed within tolerance and at least one efficiency metric improved. The project page puts it plainly: the gate rules out “savings that come from getting less done.”

The search itself was large. An automated pipeline proposed 152 directions. Each ran as its own loop: propose, implement, review by a separate reviewer, validate. Training used 535 environments, mostly mined from real GitHub issue-and-PR pairs. EdgeBench stayed held out and never fed back into the search. About one in forty ideas survived.

You don’t need 152 automated research loops to use this. You need the two gates. That’s an eval discipline, and it’s the same one we argue for in designing evals for AI agents. Write down what “still works” means before anyone touches the harness. Otherwise every cost change gets judged on vibes and a dashboard that only shows the bill.

What are the four changes that survived?

Each targets a different source of replayed tokens. The SoL-Pi repository ships all four as opt-in extensions, off by default. Its conservative example config turns on only the first and third.

Fuse an edit with its check

An agent typically edits a file, then spends a separate turn running the test or linter. Each turn replays the full context. Action Fusion lets the edit call run its follow-up validation command in the same tool call. One turn disappears per edit.

NVIDIA’s projection from recorded trajectories: about 11% fewer turns and 11.5% fewer tokens. That number is a counterfactual from replaying trajectories, not a measured rerun. Treat it as an estimate until someone reruns it.

Compact only when it pays

Compaction shrinks context, but it also rewrites the prompt cache, and cache writes cost more than reads. Online Context Compact treats finished plan steps as candidate compaction points. It then estimates the remaining requests from the pace so far. It compacts only if projected savings beat the cache-rewrite cost. Each later compaction needs a bigger margin.

This is the one we’d copy first in spirit. Many harnesses compact only near the context limit. By then the agent has already replayed the bloated context for dozens of turns. SoL-Pi keeps a near-limit fallback too. The difference is the earlier, cost-gated compaction at task boundaries, which is cheap to reason about.

Archive large tool output and keep a handle

Big tool results, such as a long file read or a search dump, sit in context forever. ObservationPack archives any result over 10 KiB locally. The first two requests see it in full. From the third request on, the model gets a stable handle, the original size, and a short head-and-tail excerpt. It can recall exact pages through the handle when it needs them.

In a paired EdgeBench run on 11 tasks, the provider bill fell about 24% and the normalized score rose about 23%. That’s a small sample. But it makes sense that less noise in context can help the model, not just the bill.

Let a cheap model read the logs, then check its quotes

Long build and test logs are mostly noise around a few lines that matter. The Evidence-Preserving Reducer sends logs of 4 KiB or more to a cheaper model, which writes a short receipt. A deterministic verifier then checks it. The exact quotes must appear in the archived log, and the source hash and exit status must match. If anything fails, or if it suspects credentials, the frontier model gets the original log.

That verification step is the pattern worth stealing for any cheap-model summarization. Summaries hallucinate. Exact-quote checks against the source are cheap, and they turn “trust the small model” into a verified claim.

One caution from the README: the reducer sends log content to a remote model. Logs often carry secrets and internal hostnames. Treat it like any other egress path, as we argue in keeping agent credentials outside the sandbox.

What the savings cost

Some capability, and NVIDIA says so. Each mechanism passed within its own tolerance. But small losses add up when you stack four changes. The combined harness kept about 94% of Pi’s average EdgeBench score on both GPT-5.6 Sol and Claude Opus 5.

Other numbers point the same way. On Terminal-Bench 4, SoL-Pi solved 15 of 63 tasks for $211. Pi and native Codex each solved 18, for $286 and $272. Cheaper per run, but fewer problems solved. Whether that’s a good trade depends on what a missed task costs you. On EdgeBench with GPT-5.6 Sol, though, SoL-Pi’s score beat the native Codex harness.

The swarm result is the most interesting caveat. NVIDIA ran a coordinator with 20 worker agents on Anthropic’s performance take-home, where fewer cycles is better. SoL-Pi workers reached 1,127 cycles for $60. Pi workers reached 1,366 cycles for $82. A single Sol agent, working alone, reached 1,333 cycles for $39.

So the SoL-Pi swarm beat the lone agent on score, at about 53% more cost. The Pi swarm didn’t beat it at all. Each condition ran once, so don’t over-read it. Still, before you tune 20 agents for efficiency, check whether you need 20 agents. We made a similar point about agent harnesses for long runs: structure should earn its place.

Other limits the authors state:

The mechanisms were tuned on GPT-5.6 Sol trajectories. They trigger less often on Opus 5.
Costs are API-equivalent estimates, not real bills.
EdgeBench used 51 of its 134 public tasks.
The hourly-savings figures ($8.75-$13.50 per hour against native Codex and Claude Code) assume one professional researcher working on a single problem.

Should your team adopt SoL-Pi?

Probably not as-is, unless you already run Pi. The value for most teams is the method and two or three of the mechanisms, applied in whatever harness you use.

Here’s how we’d sort it:

Building your own agent product. You own the harness, so all four ideas are on the table. Start with output archiving and gated compaction. They need no extra model and no new egress path.
Running off-the-shelf coding agents. You can’t change the internals of Claude Code or Codex. You can still shape what goes into context. Pipe long test output to a file and have the agent grep it. Combine edit and test into one script. Split long tasks at natural boundaries. SoL-Pi’s README also has an example that hands work from Claude Code or Codex to SoL-Pi.
Not yet measuring capability. Don’t start here. Cost cuts without a capability floor are how agents get cheaper and worse at the same time. Build the eval first. Our post on replaying production agent traces covers one way to get an eval set from real runs.

When is this the wrong project? When your runs are short. NVIDIA notes that long trajectories make token waste measurable, and shorter benchmarks hide it. If most of your agent tasks finish in a few minutes, replay costs stay small. Model choice and caching config will matter more.

How to start this week

Pick one agent workflow that runs long: a migration, a test-fixing loop, a multi-file refactor. Then:

Pull token usage per request from your traces or provider logs. Split it into input, cache read, cache write, and output. Major provider APIs return these fields in each response’s usage data.
Find the biggest single tool results that stay in context. Anything over 10 KiB that’s replayed for 20+ turns is a candidate for archiving.
Write the capability floor down before changing anything. Name the tasks, the pass criteria, and how much regression you’ll accept.
Make one change, rerun the same tasks, and compare both columns: tokens and pass rate.

If the tokens drop and the pass rate holds, keep it. If the pass rate drops, you’ve learned what that saving costs, which is more than most teams know about their agent bill.

Frequently Asked Questions

What is SoL-Pi?

SoL-Pi is an open-source, MIT-licensed extension for the Pi coding agent, released by NVIDIA’s NVlabs. It adds four opt-in mechanisms that cut token traffic on long agent runs. An automated research pipeline found them, then a capability gate filtered them.

Agent loops send the whole conversation back to the model on every turn. Prompt caching makes those rereads cheaper, but the tokens still count. On SoL-Pi’s long-horizon benchmark, cache reads were about 99% of Pi’s token traffic.

Does compaction always save money?

No. Compaction rewrites the prompt cache, and cache writes cost more than cache reads. SoL-Pi compacts only when projected savings over the remaining requests beat the rewrite cost. Even so, its cache-write traffic more than doubled.

Can I use these ideas with Claude Code or Codex?

Partly. You can’t change their internals, but you can control what enters context. Write long logs to files and grep them, fuse edit-and-test steps into scripts, and split long tasks at natural boundaries.

Spending more on agents than you can explain? Our AI cleanup work starts with exactly this kind of trace review.

Sources

Haozhe Liu, Tian Ye, et al., SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness, arXiv:2609.20519, NVIDIA, September 17, 2026.
NVIDIA NVlabs, SoL-Pi project page (retrieved October 19, 2026).
NVIDIA NVlabs, SoL-Pi repository and README (retrieved October 19, 2026).