The Stateful Inference Revolution: Building Agents That Learn While They Work
I have spent a lot of time thinking about the gap between a model that predicts text and an agent that actually gets work done. Usually, we treat the model as a static oracle. You give it a prompt, it gives you a response, and the transaction is over. If the model fails, we collect the failure, go back to a training cluster, fine-tune a new version, and redeploy it weeks later.
That is not an agent. That is a stateless function.
The recent open-sourcing of Reef by the Human-Agent-Society marks a transition toward something more interesting: the move from stateless prediction to stateful evolution. We are moving toward a world where the infrastructure does not just serve the model, but actively manages its growth through live interaction.
The death of the static model
In the traditional machine learning pipeline, inference is the finish line. It is the point where the value is extracted and the process ends. But for an agent to be effective in a complex, changing environment, the end of the pipeline must become the beginning of a new loop.
This is the core thesis of Reef. Unlike traditional reinforcement learning infrastructure that focuses on model training (like Slime or veRL), Reef is inference-first. It treats every interaction—every trajectory, every tool call, every user correction—as a structured experience stream.
Instead of waiting for a massive, periodic retraining event, the system captures these live traces to evolve the agent in real time. This is the shift from “inference as a service” to “inference as an experience stream.”
The dual nature of an agent
We often talk about “the model” as if it is the entire agent. In reality, an agent is a dual entity: it is the combination of the underlying model weights and the execution harness.
The harness is the logic that manages the agent’s tools, its memory, its prompts, and its orchestration. If you only update the weights, you are leaving half of the intelligence on the table. If you only update the prompts, you are hitting the ceiling of what the model can actually do.
True self-improvement requires the joint evolution of both. Reef handles this by using Cordis as a training backend to evolve the harness alongside the model. When the harness learns that a specific tool sequence is more efficient for a certain task, that knowledge is captured in the experience stream and baked back into the system.
The challenge of stateful inference
This approach introduces a new class of systems engineering problems. We can no longer treat inference as a simple, isolated operation. We are talking about stateful inference.
This means the infrastructure must manage an append-only release chain of artifacts. You aren’t just deploying a new container; you are managing a continuous flow of updated weights, refined prompts, and new skills.
There are also significant technical hurdles:
- Off-policy staleness: As the agent evolves, how do you ensure that the data being collected remains relevant to the current version of the agent?
- Session merging: How do you integrate experience from thousands of concurrent, diverging sessions into a coherent update?
- Performance regression: Continuous evolution carries the risk of degradation. An agent might learn a new skill but lose a fundamental one in the process. Reef attempts to mitigate this with candidate evaluation, but the risk is inherent to the architecture.
Why decoupling is the wrong path
For years, the industry has leaned into decoupling: separate the inference server from the training cluster, separate the model from the application logic, and separate the data from the deployment.
While decoupling is great for scale and simplicity, it is a barrier to continual learning. If the inference engine does not own the experience, and the training loop does not own the agent, the feedback loop remains broken.
To build agents that actually learn while they work, the infrastructure must own the experience, the agent, and the updates end-to-end. We are moving toward a paradigm where the software is no longer a static product, but a living process that grows more capable with every interaction.