DEV Community

Cover image for Don't Trust the Agent. Trust the Factory.
Remo H. Jansen
Remo H. Jansen

Posted on Edited on

Don't Trust the Agent. Trust the Factory.

The path to reliable agentic engineering isn’t waiting for models to stop making mistakes. It’s building a delivery system that catches those mistakes, limits their impact, and learns from them.

When engineers hear that we’re building an agentic factory, many picture the same end state: work goes in, an agent handles everything, and software comes out with no human involvement.

It’s understandable that this sounds unrealistic. Agents make mistakes. They misunderstand requirements, miss edge cases, and can produce tests that confirm the wrong behavior. If a factory means trusting an agent to get everything right, then skepticism is warranted.

But this week I realized we have the problem backwards.

We’re not building the factory because we trust the agents. We’re building the factory so we can trust them.

Without a factory, every change needs a fresh act of faith

An agent can produce a plausible change quickly. But if there’s no reliable way to check that change, someone has to inspect it all manually. The agent speeds up implementation, then moves the bottleneck to review.

And every mistake makes the next change harder to trust. Engineers ask for more scrutiny, more caution, more proof from the agent. The burden falls on people to assess each change from scratch.

That doesn’t scale. Nor does it make good use of what agents can do.

The answer isn’t to pretend agents won’t make mistakes. It’s to build a system that gives us evidence about each change, controls how far it can go, and responds when that evidence turns out to be incomplete.

A factory is a trust system, not an automation finish line

By factory, I mean the connected delivery system around a change: from expressing what should happen, through verification and deployment, to production feedback and recovery.

It’s more than a CI pipeline. And it isn’t a promise that a green build means the software is correct.

A change should earn its way toward production through evidence appropriate to its risk. Requirements describe the intended behavior. Tests check that behavior—not just the agent’s interpretation of it. Static analysis and builds catch other classes of problems. Sensitive changes can require additional review or controls. Staged rollouts, production monitoring, and rollback paths limit the consequences if something still gets through.

No one check proves a change is safe. The point is that we don’t have to rely on the agent’s confidence—or a person’s intuition—alone.

The factory has to learn when something gets through

A factory won’t catch everything. Tests can encode the wrong expectation. Monitoring can miss a new failure mode. Policies can leave gaps.

So the important question isn’t only, “Did the checks pass?” It’s also, “What do we do when a mistake escapes?”

Without a learning loop, the answer is often: ask people to be more careful next time. With one, a defect can leave the system stronger than before:

  • A bug becomes a regression test.
  • A recurring production symptom becomes an alert.
  • A risky kind of change gets a new policy, review step, or rollout safeguard.
  • A failure that once required manual investigation becomes easier to detect and recover from.

And that improvement benefits the next agent as well as the next engineer. The lesson isn’t trapped in one person’s memory or one code review.

Trust doesn’t mean handing over every decision

Building a factory doesn’t mean removing people from the process. It means using human judgment where it matters most.

People still define what the software should do, decide which risks are acceptable, and improve the system when real-world outcomes expose its blind spots. Some low-risk, well-understood changes may need little intervention. Changes with greater potential impact may call for deeper review or a more cautious rollout.

That’s a more useful goal than either extreme: manually scrutinising every line an agent writes, or treating a passing test suite as permission to ship anything.

The aim is not blind trust in agents. It’s earned, bounded trust in a process—one that can show why a change is ready, constrain what happens if it isn’t, and improve when it gets something wrong.

I’ve written before about tests as the feedback loop that lets agents move quickly and how AI changes the economics of software verification. This is the broader operational version of that argument: tests matter, but trust comes from the whole loop, including what happens after deployment.

The factory isn’t the idealistic end state of agent adoption. It’s what makes responsible agent adoption realistic in the first place. We don’t wait for agents to become infallible. We build a system in which their mistakes are more likely to be caught, less damaging when they aren’t, and useful to learn from.

Top comments (2)

Collapse
 
lin7c profile image
xcl •

This reframes the goal in a useful way. The line I would underline is that a change should earn its way toward production through evidence appropriate to its risk. That is a much more workable standard than either "review every line" or "trust the agent," because it admits that a documentation fix and a database migration should not have to clear the same bar.

The learning loop is the part I think teams most often skip. Turning a bug into a regression test helps, but I have seen teams stop there without asking why the existing checks missed it in the first place. The more valuable step in your list is the next one: a risky kind of change getting a new policy, review step, or rollout safeguard, because that changes what the next change has to prove.

One question on calibrating this in practice: when the evidence for a change is expensive to produce, how do you decide between accepting weaker evidence plus a tighter rollout, and blocking the change until stronger verification exists? And who makes that call in your factory, the author, a reviewer, or a policy that is set in advance?

Collapse
 
remojansen profile image
Remo H. Jansen •

I think the end goal is a combination of feature flags, A/B testing, progressive rollouts, verification debt (a score of level of risk per change) and some polocy that combines all of this with real time analytics.