Clarion Street

Beyond the Prompt: A Self-Hosted Agent Stack That Survives Production

·9 min read

I have read most of the market takes on agentic infrastructure, and they all stop one step short of useful. The story has been told from the vendor side already: the harness is the product, evaluation is consolidating, frameworks are converging. It is all true and none of it tells a builder what to do. The thing I could not find written down anywhere is the actual list: which open-source parts to assemble, and which specific failure each one is there to catch.

This is that list. It is opinionated, and it is aimed at a solo developer or a three-person team that wants to run the whole thing self-hosted without paying a platform vendor to do it for them. Every part I name is MIT or Apache licensed, so the licensing does not get in the way of running it yourself. Whether your data actually stays on your machines is a separate decision, made by where you point the model, and I get to that.

The reason the prompt stopped being the hard part is measurable. You.com’s 2026 open-source survey puts it plainly: “the best teams building agents in 2026 aren’t writing prompts, they’re architecting systems.” Uvik’s framework comparison puts a number on it. The same Claude Opus 4 scores 64.9% on GAIA in one scaffold and 57.6% in another, and the gap between a bare model and a well-built scaffold reaches around 30 points. The model did not change between those runs; the wiring around it did.

That is the whole argument for caring about the harness in one paragraph, and I will not relitigate it. This site has made the market version of that case already, so what follows is the assembly.

The three ways an agent fails

Every production agent failure I have seen collapses into one of three buckets.

A tool call fails. It returns a malformed payload, times out, or does something with a side effect nobody wanted. The bluntest version of this in the sources is You.com’s line that most agents fail because tools fail. I will note up front that this is a vendor’s aphorism, not a peer-reviewed finding, but it names the right failure.

The agent loses its place. It runs for twenty minutes, calls six tools, waits on a human, and then the process restarts and it has forgotten where it was. This is state loss.

The output is right on Tuesday and wrong on Wednesday, for the same input, and you cannot tell which way the needle moved. This is non-determinism, and it is the one that quietly kills confidence.

None of these is fixed by a better prompt. Each is handled by a specific piece of software.

The parts, and what each one absorbs

The runtime: EngineX or Pydantic AI

You need something to run the agent, and two open-source options sit at opposite ends of the field.

Pydantic AI is the safe pick. It is MIT licensed, model-agnostic, and, in its own framing, typed end to end. The selling point is that your agent’s inputs and outputs are Pydantic models, so a malformed tool response fails at the type boundary instead of three steps later. It sits at roughly 20,400 GitHub stars and had commits landing within hours of me writing this. It is part of a wider Pydantic stack, Harness, Logfire for observability, an AI gateway, evals, and a typed graph library, so you can grow into those without switching ecosystems.

EngineX is the other end. It is an MIT-licensed goal-driven agent runtime that bundles the things you would otherwise wire yourself: graphs, human-in-the-loop pause nodes, MCP tool servers, a credential vault, and a web dashboard. It is designed for self-hosted installs in a client VPC, stores credentials encrypted locally, and has a local-LLM path through Ollama that needs no API key. That last bit is the attractive part for anyone who wants to keep inference at home.

I have to be blunt about the trade. EngineX has 8 stars, 8 forks, and 109 commits, with its last commit dated August 2026. It is a young project, not one that has survived much production use. I am not presenting it as a proven alternative to Pydantic AI or LangGraph. I am presenting it as proof that the self-hosted runtime category exists, and as something to watch or contribute to rather than bet a job on. If you need maturity today, start with Pydantic AI and add the other pieces yourself.

State: checkpoints

The part that fixes state loss is persistence, and LangGraph names it precisely. It splits persistence into two things. Checkpointers persist the state of a single thread, so an agent can resume where it stopped, back up to a prior step, or pause for a human. Stores hold cross-thread memory that outlives any one conversation.

The gotcha that matters for self-hosting is that the in-memory checkpointer is lost on restart. If you run in a process that restarts on deploy, an in-memory saver is a demo feature. Production needs a persistent one, and the docs point to PostgresSaver or SqliteSaver. Postgres or SQLite is the entire infrastructure requirement. You do not need a vendor for this.

Tools: MCP

Tool failure is partly a wiring problem and partly a reliability problem, and it is worth keeping the two apart, because MCP only addresses the first. It is an open-source standard for connecting an AI application to external systems, and its own framing, a USB-C port for AI applications, is the cleanest description I have found. It is now stewarded by the Linux Foundation, with backing from Anthropic, OpenAI, Google, Microsoft, AWS, Cloudflare, Block, and Bloomberg. Every framework in Uvik’s survey ships MCP support natively or through an adapter.

What MCP buys you is interoperability. A tool that speaks the protocol plugs into any MCP-capable agent through one standard connection, so adding a tool is less bespoke integration work than writing a custom adapter. What MCP does not do is make a flaky tool reliable. Retries, timeouts, output validation, and control over side effects belong to the runtime and the tool implementation, not to the protocol. EngineX ships MCP tool servers for Slack, browser automation, the terminal, and files as a built-in path. If a specific tool is where your agent fails, MCP can reduce the integration work of swapping it, but only when the replacement server is compatible and keeps the same schema and semantics. It does not guarantee a drop-in swap that leaves the surrounding agent code untouched. And the retry and validation logic still has to live somewhere concrete.

Evaluation: Ragas and Promptfoo

A better runtime does not remove non-determinism, and evaluation does not either. What evaluation does is make the variation visible and give you something to gate on.

Ragas is the Apache-2.0 library for scoring relevance, faithfulness, and answer quality, at roughly 15,900 stars. Promptfoo is the MIT testing framework: declarative config, CI/CD integration, and red-teaming. Its README touts evals that run fully locally, but that holds only when you point them at a local model or provider; configured with a hosted provider, your prompts leave the machine like any other API call. It is the more actively maintained of the two right now, latest commit October 5, 2026 against Ragas’s February 2026, which is worth knowing if you plan to build on one long term.

The academic backing for why this matters comes from a CLEAR, a November 2025 arXiv preprint. Its measured finding is that an agent scoring 60% on a single run drops to 25% across eight consecutive runs, and that cost varies up to 50x for similar precision. Its policy finding is sharper: optimizing for accuracy alone produces agents 4.4 to 10.8 times more expensive than cost-aware alternatives with comparable performance, and multi-dimensional evaluation predicts production success much better, at a correlation of 0.83, than accuracy-only scoring, at 0.41. CLEAR is a single-author preprint, not peer-reviewed, so those are the author’s own measurements. The direction still matches what anyone who has run an agent in production already suspects: a single run is not much evidence.

The checklist

This checklist is what I actually wanted when I went looking for it. A short mapping from failure to the part that catches it, to run against before you call something production.

  • Tool failure. Route calls through MCP to standardize the connection and cut down on bespoke integration, and have the runtime validate or retry a bad response instead of passing it forward. EngineX does the validation and retry with its event-loop nodes; Pydantic AI catches malformed output at the type boundary. MCP standardizes the connection, not the tool’s reliability, and a swap is only clean when the replacement keeps the same schema and semantics.
  • State loss. Use a persistent checkpointer, Postgres or SQLite, not an in-memory one, and a store for anything that must outlive the thread.
  • Non-determinism. Run eval in CI, score more than accuracy, and treat a single passing run as a demo rather than a result.

If a part does not map to one of those three rows, I would question whether it earns its place. Checkpoints, retries, and an eval harness each exist to catch a specific, predictable failure, and together they are what separate a demo that works once from a system that survives.

Bottom line

I want to be clear about what I am and not claiming.

What I am claiming is narrow. The harness the industry treats as the real product surface can now be assembled by one person from MIT and Apache licensed parts, self-hosted, without a vendor. The pieces are Pydantic AI or EngineX for the runtime, the LangGraph persistence for state, MCP for tools, and Ragas or Promptfoo for evaluation. None of it requires a platform subscription.

Keeping your prompts local is possible but not automatic, and the license does not do it for you. Pydantic AI is model-agnostic and will happily call a hosted API unless you point it at a local model endpoint. Ragas and Promptfoo can also be configured with external model providers. If you want everything to stay on your own hardware, the model has to run there too. EngineX’s Ollama path is one documented way to do that, and you then have to point the evaluation tools at that same local model.

What I am not claiming is that any of it is easy, or that any one framework is the answer. Uvik’s own conclusion is that every framework choice can be correct, and that the one wrong move is choosing by star count. The numbers I cited are attention, not adoption, and EngineX in particular is too young to bet on. CLEAR is a preprint.

Sources

  1. EngineX GitHub repo — EngineXV/engineX
  2. Agentic Open-Source Tools 2026 — You.com
  3. Agentic AI Frameworks: 2026 Production Comparison
  4. Pydantic AI GitHub repo — pydantic/pydantic-ai
  5. Ragas GitHub repo — vibrantlabsai/ragas
  6. Promptfoo GitHub repo — promptfoo/promptfoo
  7. Model Context Protocol — official site
  8. LangGraph: Persistence — LangChain docs
  9. CLEAR: Beyond Accuracy — arXiv 2511.14136