Agentic Runtimes Went Open Source, and the Rack Became a Category

October 14, 2026·8 min read

Between October 12 and 13, four things happened that I have not seen happen in a single week before. NVIDIA open-sourced the Run:ai platform in full. StreamNative released the Orca Agent Engine, a runtime for managed agents. AIBrix, the vLLM project’s Kubernetes inference stack, shipped a release built around session-aware routing. And at the OCP Global Summit, AMD, Intel, and NVIDIA each described a CPU-dense rack for running agents. The software that runs agents went open source, and the hardware it runs on was named as a category, in the same 48 hours.

I have made the general argument before. The model stopped being the constraint, and the interesting work moved to the layer around it. That case is already on this site, in the bottleneck post, the durable execution post, and the control plane post. I am not going to re-argue it. What changed this week is specific, and it is the part worth writing down: the runtime went open source, and the rack began to be treated as an open, heterogeneous category, in the same week.

The runtime opened first

NVIDIA’s move is the biggest. Run:ai was the GPU-orchestration company NVIDIA acquired, and the product stayed closed through the deal. On October 13 NVIDIA released the full platform as open source under the DSX AI Factory umbrella, split into four pieces you can adopt separately. KAI Scheduler, the Apache 2.0 scheduler NVIDIA has contributed to the CNCF as a Sandbox project. The GPU Fractioning Operator, which does CUDA 13.4-native GPU fractioning. KAI Resource Management, for organizational quotas and hierarchy. And Workload Map, a workload-management contract for Kubernetes. Eight partners are already integrating it: Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud, and vCluster.

The Orca Agent Engine, from StreamNative, the company behind Apache Pulsar, is a different kind of release. It is a declarative runtime for building, deploying, and governing agents, out as a Developer Preview under Apache 2.0, with a CLI and TypeScript, Python, and Go SDKs. Two of its design choices tell you what the founders think an agent is. Governance lives outside the agent: guardrails can only tighten, and credentials stay outside the sandbox. And every session produces an event log, which is also how the runtime resumes interrupted work. The launch post frames the workload the way a distributed systems person would. An agent needs an identity, an isolation boundary, a budget, and a durable record. Cost, the post says, is “an attribution problem before it is a price problem.” That line lands because it is true: you cannot control what you cannot attribute.

AIBrix, the vLLM project’s serving stack, shipped v0.8.0 the day before, with 443 commits from 104 contributors. The headline is Redis-backed session-key pinning. Multi-turn agent workloads now route by session affinity, prefix-cache locality, and replica load, instead of as independent requests. That is a small-sounding change that matters. A multi-turn agent is a sequence of requests that share a cache and a state; routing each one as an independent request throws that state away. I wrote about this stateful turn earlier, in the stateful inference post. AIBrix also added ModelClaim, which decouples a model’s lifecycle from the pods that host it, and ModelWarmup. The release notes say the project is moving toward “more dynamic and heterogeneous inference systems,” with encode, prefill, and decode placed and scaled independently.

The three releases are not the same product, but they are the same answer. A declarative runtime for agents, a scheduler for GPU capacity, and a router for session state each treat an agent as a workload with continuity rather than a string of stateless requests.

The rack became a category

OCP 2026 made the same point in hardware. Intel’s keynote was the most direct: agentic AI turns inference into a multi-step workflow spanning retrieval, reasoning, orchestration, tool use, and execution, and x86 CPUs sit at the center of it. Intel announced the GPU D-series, code-named Crescent Island, on Xe3P architecture with up to 480 GB of LPDDR5X and 1.5 TB/s of bandwidth in a 350 W envelope, sized for models up to 300 billion parameters on one card. It also pointed at Xeon 6+ configurations scaling past 46,000 cores per rack.

AMD answered with Helion, an open rack-scale reference design that mixes CPU-only racks, up to 32,768 EPYC cores, with GPU inference systems and a “token router” that directs each request to the right hardware. AMD plans to contribute Helion to OCP, with partner systems expected from November. Lisa Su’s numbers are company projections, not measurements, but the direction is the point: 2026 as “the year of inference,” a roughly 14x rise in agentic traffic, and a data-center CPU forecast raised above $200 billion by 2030.

Arm put it most plainly. Mohamed Awad called agents “virtual users” and argued they need their own class of CPU, split across three jobs: accelerator support, general-purpose cloud, and agentic execution. Arm published a white paper with Fujitsu, Meta, and OpenAI on benchmarking CPUs for agentic workloads. NVIDIA, on the same stage, announced a liquid-cooled Vera CPU rack with 256 Vera CPUs aimed at agent execution, orchestration, and tool calling.

OCP itself is the other signal. It is expanding from servers and racks to the whole data center: silicon, cooling, power, facilities, and the grid, with about 15,000 attendees and a membership expected to pass 700 organizations by the end of the year.

Two layers, one emerging shape

The runtime and the rack landed in the same week because they answer the same question. The case made at OCP, by Intel, AMD, and Arm, is that an agent needs two kinds of compute: GPU cycles for the inference itself, and CPU cycles for the orchestration, tool use, and execution around it. Whether that split actually holds for real workloads is still open. But the rack designs assume it does. They are built around the CPU and GPU mix rather than around raw accelerator density, and the scheduler is expected to follow.

The runtime releases line up with the software half of that argument. Orca gives an agent an identity, a budget, and a durable record. Run:ai’s KAI Scheduler and GPU Fractioning Operator parcel out the accelerator side. AIBrix’s session pinning keeps a multi-turn conversation on a warm cache. Helion’s token router and the Vera rack are the hardware half, at least on paper. None of these vendors coordinated. They answered the same question at the same time because the question had become unavoidable.

None of this is charity. NVIDIA open-sourcing Run:ai after acquiring it, and handing KAI Scheduler to the CNCF, is a play for the orchestration layer, the same way the OCP specs are a play for the rack layer. Orca’s founders say it outright: teams should be able to inspect and extend the runtime that connects them. The goal is to make one stack the default. “Open” here is a distribution strategy. It does not mean NVIDIA is walking away from its GPU or software business, and it does not mean the ecosystem is getting something for nothing.

What I am not claiming

Most of this is announced, not shipped, and much of it is vendor self-description. Four of the six sources are company announcements, and the other two mostly relay keynotes. AMD’s $200 billion figure and the 14x traffic number are forecasts. Intel’s core counts and power numbers are design targets, and the 480 GB figure is the ODM-partner maximum, not the Intel-branded card, which carries 160 GB.

“Agentic rack” is my shorthand and Data Center Frontier’s phrase, not an OCP term. AMD calls its design Helion; NVIDIA announced a Vera CPU rack. The phrase is useful, but treat it as a category being named in real time, not a settled standard.

And none of it is proven at production scale. Orca is a Developer Preview. AIBrix’s ModelClaim, ModelWarmup, and session pinning are labeled experimental. Intel’s D-series is still in customer sampling. Helion partner systems are “expected” from November. The honest read is that the software layer went open and the hardware layer got drafted on paper, all in one week, and the hard part, running real multi-agent workloads on this hardware with these runtimes, has not happened yet.

I do not take that as a reason to ignore it. The last time I watched a layer standardize this fast was the model layer, and the downstream effect there was real. The difference now is that two things below the model are standardizing at once, the open-source runtime and the vendor-drafted rack, and the people drafting them are the ones who sell the hardware and the clouds. I am watching whether they settle on the same shape, the one the OCP keynotes sketched, with CPUs carrying the orchestration around GPU inference. If they do, the agentic stack will have a shape, and this week is when the outline was drawn.

The runtime opened first

The rack became a category

Two layers, one emerging shape

What I am not claiming

Sources