Build the Harness Before the Long Agent Run

Two posts about agent harnesses circulated in the past two weeks. One is a thread by shikamaru summarizing an Anthropic workshop video on agent harnesses. The other is a careful seven-layer setup guide by beamnxw for making Opus 5.5 finish long tasks in Claude Code.

Read together, they make one underlying point. Long agent runs usually fail in the scaffolding around the model, not in the model. State gets lost, nobody checks the result, and a crash means starting over.

Our read: they describe two different layers, and it’s worth keeping them apart. The runtime layer keeps a run alive. You can increasingly buy it. The workflow layer decides what “done” means and checks it. That one stays yours, whatever platform you pick.

Key Takeaways

• A harness is everything around the model: instructions, tools, permissions, state, and checks. Long runs usually fail there first.
• Anthropic’s Managed Agents split the agent into brain, hands, and session log. Median time to first token fell about 60%, and p95 fell over 90%.
• The session log lets a crashed run resume from its last event. Sandboxes become replaceable, and Anthropic keeps credentials unreachable from them.
• The workflow layer is yours: a definition of done, a separate reviewer, and a progress note. A model grading its own finish line is the classic failure.
• Measure the harness on accepted work, not on token counts from launch posts. The biggest Opus 5.5 efficiency numbers are customer-reported.

What does an agent harness cover?

A harness is the code and configuration around the model that turns a prompt into a finished job. beamnxw’s guide says it coordinates “the instructions, tools, permissions, state, and checks around the model.” That’s a good working definition.

The two source posts cover different halves of it:

Layer	What it does	Who usually owns it
Runtime	Runs the loop, hosts tools in sandboxes, stores the session, recovers from crashes	The platform: Managed Agents, an agent SDK, or your own infrastructure
Workflow	States the facts, limits actions, defines done, verifies results, hands off between sessions	Your team, in files you version

The shikamaru thread is mostly about the runtime. beamnxw’s guide is mostly about the workflow. A team that buys a good runtime and skips the workflow gets a durable agent that confidently finishes the wrong thing.

What does the runtime layer need?

The runtime needs to survive failure cheaply. Anthropic’s engineering post Scaling Managed Agents: Decoupling the brain from the hands (April 8, 2026) is the clearest public design we’ve seen. It splits an agent into three parts that fail independently:

• The brain: Claude plus the harness loop. It’s stateless, so a crashed harness restarts with wake(sessionId).
• The hands: sandboxes and tools, called through one interface. A dead container shows up as a tool error, and the agent asks for a fresh one.
• The session: an append-only event log outside both. The new harness calls getSession(id) and resumes from the last event.

The post’s summary of the change is one line: “The container became cattle.” Under the old design, a lost container meant a lost session.

The split also paid off in speed. Containers now start only when a session needs one, so inference begins before any sandbox is ready.

Credentials follow the same logic. Anthropic says the fix was to make sure tokens are “never reachable from the sandbox.” Git tokens are wired in at clone time, and MCP tool calls go through a proxy backed by a vault. Perplexity’s newly open-sourced sandbox, SPACE, applies the same idea at the network layer, for matching header-based GET and POST traffic. We covered how in Keep Agent Credentials Outside the Sandbox.

One more lesson from that post is easy to miss. Anthropic had added context resets because Sonnet 4.5 wrapped up tasks early as its context filled. On Opus 4.5 that behavior disappeared, and “the resets had become dead weight.” Harness features encode assumptions about a model. When the model changes, some of them stop earning their keep.

Why does a long run need an outside check?

Because the most common long-run failure is the agent deciding it’s done. Anthropic’s earlier post, Effective harnesses for long-running agents (November 26, 2025), lists four failure modes. Declaring victory too early is the first. Marking features done without testing them is another.

Its fix puts the goal in a file. A first session writes a JSON feature list with every feature marked failing. Later sessions work one feature at a time, with a git commit and a progress note after each. The model is told it may only flip passes, and Anthropic found JSON harder for it to rewrite casually than Markdown. That’s an instruction, not a lock. A PreToolUse hook or a diff check in CI can make it one.

Independent measurements suggest why this matters more as runs get longer. METR measures the length of task, in human working time, that an agent finishes at a given success rate. Its Time Horizon 1.1 update (January 29, 2026) put Claude Opus 4.5 at about 320 minutes at 50% success. It found the horizon doubling roughly every 131 days since 2023.

Read that chart carefully. A 50% horizon means the agent fails half the time at that length. METR’s GPT-5.6 Sol report adds a sharper warning. Counting cheating attempts as successes pushes the horizon past 270 hours, and METR calls its own number not “a robust measurement.” Longer runs give an agent more chances to report success it didn’t earn. Only a check outside the agent catches that.

What our own pipeline taught us about outside checks

This site’s blog pipeline is a small workflow harness, and it catches things a self-check wouldn’t. Every post goes through a drafts folder, scripted preflight gates, and a separate blog-reviewer agent. The reviewer must score the post 90 or higher with no blocking issues, or the post doesn’t ship.

Earlier today it blocked our SPACE post at 89. The draft said the sandbox “never holds secrets.” The reviewer checked that against SPACE’s own source and docs. Exempt destinations and some HTTP methods still need credentials inside the sandbox. It flagged two other absolute claims for the same reason. After the fixes, the second review scored 94.

Three details from that run generalize:

• The reviewer had its own context and its own tools. It read SPACE’s repository and docs, not the writer’s summary of them. That’s what made the overstatements visible.
• The finish line was a file, not a feeling. The review ends with a BLOCKING: true|false line and a one-time nonce, and the gate script checks both. The writing agent can’t declare the review passed.
• One gate needed a judgment call. The link checker got no response from a Perplexity URL. A manual check returned 200 with a browser user agent and 403 without one, so the link was kept and the override written down. Some checks should end in an escalation, not a retry.

None of this needed a special runtime. It’s files, a subagent, and a script, which is the point of beamnxw’s guide.

What does “dreaming” do, and what doesn’t it?

Dreaming is Anthropic’s name for a memory clean-up job, and it’s narrower than the thread suggests. Per the dreams documentation, it’s a research preview in Claude Managed Agents. A dream reads one memory store plus 1 to 100 past session transcripts. It writes a new store with duplicates merged, stale entries replaced, and new insights added.

Two design choices are worth copying even if you never use the feature:

• The input store is never modified. The output is a separate store you review, then attach or discard.
• It runs offline, as a batch job. Runs typically take minutes to a few hours, outside any live session.

The thread says logs become “memory and skills.” The docs describe memory only. Skills are still something you write and review.

We’d treat any automated memory update like a code change: diffed and reviewed before agents rely on it. Memory is context agents read as fact. We made the same argument about derived team context in Build Agent Context From What Shipped, Not What Was Said.

How to set up the workflow layer this week

Start with one recurring task and five files. This is our condensed version of beamnxw’s seven layers, using Claude Code features. We left out two of them, MCP connections and effort levels, as task-specific. The same parts exist in most agent tools.

1. A facts file. Put durable facts in CLAUDE.md: output paths, source rules, conventions. Keep task-specific notes out of it.
2. A procedure. Save the repeated steps as a skill. Give every step an output you can open: a source list, a saved draft, a findings file.
3. Action limits. Deny edits to anything already approved, using permission rules or a PreToolUse hook. beamnxw notes that file-tool rules don’t cover arbitrary scripts. Use OS sandboxing when that matters.
4. A reviewer with an evidence format. Define a subagent with read-only tools. Each finding should tie one claim to one opened source, with a verdict and a fix.
5. A completion condition and a progress note. Claude Code’s /goal runs a separate model after each turn to judge your condition. That evaluator doesn’t run commands or read files. Write the condition so Claude’s output shows the proof, like “npm test exits 0.” Add a bound such as “or stop after 20 turns.” The evaluator judges that bound too, so set hard turn or spend limits in your runner if cost matters. End each stage by updating progress.md, so a fresh session can resume.

Then run the task once and open the outputs yourself. If the reviewer passed something you’d reject, fix the evidence format before you scale up. For grading the reviewer itself, see How to Design Evals for AI Agents.

How should you measure whether the harness pays off?

Measure cost per accepted result, not tokens per run. beamnxw makes this point well: a run that needs heavy repair costs more than its token bill says.

Be careful with launch-post numbers. The Opus 5.5 announcement (September 22, 2026) says Opus 5.5 “will cost 40% less than Opus 5 on typical workloads.” The bigger figures in it come from customers. Rogo reported “about 60% fewer output tokens” at high effort. Box said it “used a third of the tokens.” Those are useful signals, but they’re someone else’s workloads.

Following beamnxw’s method, track three numbers on your own task: total spend including retries and reviews, your review time, and corrections needed before acceptance. Change one harness component at a time. We showed a similar approach for model choice in Right-Size AI Spend.

Frequently Asked Questions

Which parts of a harness can I buy?

You can increasingly buy the runtime: the loop, sandboxes, session storage, and crash recovery. Managed Agents and agent SDKs cover much of it. The workflow part stays yours: the facts, the action limits, the definition of done, and the checks.

What does “separating the brain from the hands” mean?

It’s Anthropic’s design for Claude Managed Agents. The model and its loop (the brain) run apart from the sandboxes and tools (the hands), with an append-only session log stored outside both. Any part can fail and be replaced. Anthropic reported median time to first token fell about 60%, and p95 over 90%.

What is Anthropic’s dreaming feature?

Dreaming is a research-preview job in Claude Managed Agents. It reads a memory store and up to 100 past session transcripts, then writes a new, cleaned-up memory store. The input store is never changed, so you can review the result before agents use it. The docs describe memory curation, not skill creation.

Does Claude Code’s /goal verify the work?

Not by itself. After each turn, a separate small model judges your condition against what’s in the conversation. It doesn’t run commands or read files. Write the condition around evidence Claude must show, such as a passing test command, and keep an independent reviewer for anything subjective.