Clarion Street

The Harness Is No Longer Hand-Built: Meta-Orchestration and the Rise of Self-Evolving Agents

·6 min read

The harness is the least glamorous part of building an agent and the most important. It is the layer around the model: the planner, the tools, the skills, the memory. A model by itself answers a question. The harness is what turns it into something that does work, and today that layer is assembled by hand. Teams pick a framework, wire up tools, write prompts, add memory, and then spend most of their time fighting the harness rather than the model. In most production systems the harness, not the model, is what caps how good the final agent gets. I traced that manual layer when the infrastructure story was still new. This piece picks up from there.

Two things shipped in the last few weeks that automate that layer away from opposite directions, and together they are a bigger change than either one alone.

Raven is the version that automates from below. EverMind released it as an Apache 2.0 open-source multi-agent system, and it calls itself, in its own words, “the harness of harnesses” and “a self-evolving multi-agent orchestration ecosystem.” The claim is that Raven constructs and evolves the harness itself: the planners, the tools, the skills, the memory. A Host Agent breaks a goal into a directed acyclic graph of subtasks, matches each one to a specialized agent, coordinates the dependencies, and integrates a single deliverable at the end. It ships four built-in agents (research, code, design, oncall) plus presets for 13 external CLI agents including Claude Code, Codex, OpenCode, and Hermes Agent, and it carries memory across sessions.

The evolution part is explicit rather than implied. Two components, Evolver and Curator, diagnose failures, propose changes to the harness, test them, and accept only the edits that verify. The project is pre-alpha and the self-development module is marked experimental, so treat the claim as a direction, not a working product.

The technical report (arXiv:2609.33439) reframes the central question. The old question was how to engineer a stronger harness for one domain. The new one is how to autonomously construct specialized harnesses, improve them through experience, and orchestrate them across domains. Each executable model-plus-harness pair becomes a composable unit of intelligence. This is the same ground as the framework spectrum I laid out last week, taken one step further: instead of picking an arrangement, the arrangement assembles itself.

OpenAI’s Agents API is the version that automates from above, and it shipped first, on September 10 in public beta. The pitch is the opposite of Raven’s. “OpenAI hosts and maintains the harness.” You specify the task, the model, the tools, and the compute environment in one API call, and the provider handles context compaction, tool search, sub-agent orchestration, and future upgrades. “We maintain and continuously improve the harness alongside our models,” the announcement says. You can run it in OpenAI’s sandbox, on your own compute, or with a list of ecosystem partners.

The two approaches automate different halves of the same job. Raven automates construction. OpenAI automates maintenance. Either way, the harness stops being something a developer hand-builds and owns.

The demonstrations are the part people will quote, so they are worth a close look. Raven’s THRESHOLD case ran 42 rounds of a plan, develop, verify cycle over about four days and produced a playable Godot 4 first-person shooter with an arena boss fight, plus a website, a poster, and a presentation, all from a human brief. A second case, Raven RSI, ran 172 training runs across 7 rounds in nanochat pre-training without a crash, cutting val_bpb by 5.8% inside a fixed 20-minute single-GPU budget. Every benchmark figure is EverMind’s own run on benchmarks it chose, and none of it has been independently replicated. The headline scores, 76.5% on the DeepResearch Mixed aggregate and a 0.8762 Pass@1 on DataAgentBench, come from the same self-reported source.

The part I keep coming back to is the audit problem, not the demo.

We have spent a string of posts here on the difference between a trace and an audit trail, and on the verification gap that agentic workflows opened. The short version: a trace records what an agent did, but proving a decision was justified is the harder problem, and it is the one an auditor actually cares about. A self-evolving harness makes that problem worse in a specific way. When a human builds the harness, the human can explain the planning layer, because the human designed it, reviewed it, and pinned it down. When the system builds and revises its own harness, the planning layer changes from run to run under the system’s own control. You now have to attest to what a system did when the system also wrote the code that decided what to do.

That is a compliance problem before it is an engineering one, and it is the part of this story our readers should care about. The topology-as-optimization work I covered earlier treated the arrangement as something you optimize. Raven goes further and treats the arrangement as something that evolves on its own. The audit thread has been consistent about one thing: legibility matters. A harness that edits itself is the least legible thing on the table. If you cannot point at the planner and say what it was and who changed it, you are left arguing that the deliverable is good, which is not the same as proving the process was sound.

There is a counterpoint worth holding onto. In the same window, Swarms shipped v16 “Overclock” with more than a hundred improvements: a typed DecisionModel, an MCPDeployer for serving agents as MCP servers, and a TreeOfThoughts. That is the manual-framework paradigm still shipping real features. The hand-built harness is not retiring. It is becoming one option among several, and for a self-hosted operator with an auditor somewhere downstream, it may stay the right one for a long time.

My read is that meta-orchestration is real and unverified, in that order. The direction is clear and I would not bet against it. The evidence is thin and entirely self-reported. For a specialized multi-agent deployment the barrier to entry drops, because a team verifies configuration, artifact acceptance, and experiment boundaries instead of building a framework from scratch. But the thing that gets automated away is also the thing that used to be your explanation for why the system did what it did. When that goes, the value moves to brief design, eval sets, and the tooling you use to decide whether to accept a result. The verification gap, which was already the hard part, gets harder.

Sources

  1. EverMind Releases Raven: An Open System That Builds and Evolves Agent Harnesses — mlllm.io
  2. Raven: The Harness of Harnesses for Composable Agentic Intelligence — arXiv
  3. Raven: The Harness of Harnesses for Composable Agentic Intelligence (full text) — arXiv
  4. Introducing the Agents API — OpenAI
  5. Swarms v16 Overclock: Decision Models Router, MCP Deployment, TreeOfThoughts — discourse.cosign.co
  6. EverMind-AI/Raven — GitHub
  7. Raven — The Harness of Harnesses — EverMind