Clarion Street

The Agentic Runtime Splits: Governed Runtimes and the Local Inference Server

·6 min read

Two Apache 2.0 projects shipped in the same week last week, and neither is a control plane. On October 6 StreamNative open-sourced Orca Agent Engine, a declarative runtime that keeps guardrails, budgets, and credentials outside the agent’s reach. On October 8 Base Compute announced Superfluid, an inference server for the case where one machine has to serve several agents at once. This site has spent the past month on the control plane story: the Kubernetes moment for AI, OpenClaw Enterprise, and the open-source sprawl fixes. Orca and Superfluid are not that story. They sit at its edges, and the edges are worth watching.

The two products have almost nothing in common on the surface. Orca is for the operator who needs an audit trail and a spending limit. Superfluid is for the person whose laptop now runs a small fleet of agents. What they share is that both treat an agent as a workload with scheduling and record-keeping needs, not as a chat feature.

Orca: governance inside the runtime

Orca Agent Engine came out of StreamNative, the Apache Pulsar company, and the lineage shows. It was introduced a year ago as the StreamNative Agent Engine and open-sourced on October 6 in Developer Preview. You declare agents as resources, then pick a harness and a model as fields in the configuration. The harness options are Claude Agent SDK, the default, along with Codex and Pi. The model options are Anthropic, OpenAI, Google Gemini, DeepSeek, and MiniMax, plus any OpenAI-compatible endpoint. Because both are fields, a routine task can run on a cheap model while harder work runs on a frontier one, without changing the harness integration.

The engine splits into five components: a Registry that stores declarations and never runs agent code, a Harness Server, the Sandbox, the AI Gateway, and the Event Store. StreamNative positions Orca as doing for agents “much like Kubernetes did for microservices,” and the architecture grew out of Pulsar Functions, the serverless compute layer the company already runs.

The governance is what separates Orca from the control-plane pattern. Guardrails are enforced outside the agent’s editable configuration, and they are scoped in layers: organization, then workspace, then agent, then session. Each rule returns allow, ask, or deny, and a lower scope can tighten a rule but never loosen one set above it. Budgets are rules too. A soft threshold asks before more spend, and a hard cap stops the agent. Orca does not quietly swap in a cheaper model to stay under the cap.

What I keep coming back to is what the agent never gets to hold. Provider credentials live in a vault and are resolved at the AI Gateway, which checks the session token and injects the secret into the outgoing call. The credential value never enters the sandbox. A session is an append-only event log, the TranscriptStore in the code, backed by Kafka, Postgres, or Pulsar. That log is how an interrupted session resumes, and it is the record you inspect afterward.

Orca is open about its limits. Delivery is at least once, not exactly once, so tools that change external state should be idempotent. Sandboxes are disposable, and unpushed edits to a repository are lost when one is destroyed. This is a Developer Preview, everything I can verify traces to StreamNative’s own posts, and there is no independent evaluation. But governance here is a property of the runtime, not a layer bolted on top of it.

Superfluid: a scheduler for one machine

Superfluid comes at the problem from the other direction. Base Compute’s opening line names it directly: inference servers were built for throughput on datacenter racks, not for “one person sharing a laptop with eight agents.” One agent summarizing your email overnight, one watching health data, one doing the shopping, while you still want to ask a question in between.

The fix is a scheduler. Interactive chat preempts batch work at the next tick, so your question does not wait behind whatever the agents are doing. Every token is written to a log before you see it, so a crashed worker or a restarted server picks up its sessions where they left off. Thirty-two requests sharing one system prompt pay one prefill. The server speaks the OpenAI, Anthropic, and Ollama APIs over llama.cpp, MLX, and BaseRT runtimes, and it can launch Claude Code, Codex, pi, OpenCode, Hermes Agent, Cline, OpenClaw, or the Ollama CLI against itself.

The performance numbers deserve a skeptical read. Base Compute measured on an Apple M5 Pro running Qwen3.8-27B at 4 bits. A chat request behind eight agents got its first token in 1.5 seconds, against 291 seconds for llama-server. A demo in the same post shows a chat answer in 0.36 seconds where llama-server takes more than ten, which is where the “30x faster” line comes from. All of it is one vendor, one machine, one quantization, and none of it is independently audited.

This is where Superfluid lands on ground the site has already covered. Sovereign AI for the individual and the hidden bloat in agent runtimes made the case for running your own models. The new part is the observation that the many-agents problem the enterprise vendors call sprawl also exists on one person’s laptop, and there it is a scheduling problem rather than a governance one.

The split

Set the two side by side and the split is plain. For the past month the story has been the control plane, a layer that watches a fleet of agents and decides what they may do. Orca and Superfluid answer different questions. Orca pushes governance down into the runtime, into the session log and the gateway that holds credentials. Superfluid pushes the workload down into the device, where the hard problem is not who is authorized but who gets the GPU next.

I think that is the useful takeaway from the week, and it is not a vendor’s talking point. The runtime is no longer one thing. It is splitting into a governed form for operators who need records and limits, and a local form for people who need their own machine to behave like a small cluster. The control plane is still where the big vendors fight, and the site will keep covering it. But the defaults for everyone else get set at the edges, and the edges are what shipped last week.

Caveats

Both products are new and unproven. Orca is a Developer Preview whose technical claims all come from StreamNative’s own posts. Superfluid’s performance table is vendor-measured on one Mac with one model and one quantization, and the “30x” number should be read as direction, not an audited benchmark. Neither has an independent evaluation, and a launch date is not the same as proof it works in production.

Sources

  1. Open-Sourcing Orca Agent Engine — StreamNative, October 6, 2026
  2. Inside Orca Agent Engine — StreamNative
  3. Introducing Superfluid — Base Compute, October 8, 2026
  4. Introducing Orca — StreamNative Agent Engine (Private Preview) — StreamNative, September 30, 2025