AI agent reliability: why a 10-agent fleet only gets you to 60 percent

A proposal landed on my desk this week asking for five freelancers to build a fleet of ten AI agents, running across four different frameworks, to handle the entirety of an engineering firm's operations. I told them not to build it. Not because AI agents do not work, but because ten of them chained together is a maths problem before it is anything else. This post is the arithmetic I showed them, and the much smaller system we recommended in its place.

What the proposal actually asked for

Ten agents. Four frameworks. LangChain, CrewAI and AutoGen among them. The stated outcome was 80 to 90 percent of all administration and project management work absorbed by the agents. They wanted their hours back, they wanted nothing falling through the cracks, and they wanted one system instead of six open tabs.

That outcome is entirely reasonable to want. The route to it was wrong in three separate ways, and each one compounds the others.

The first is sequencing. Nothing reaches production until the last automation lands. Built properly, with real security and testing, that is six to nine months before the business feels a single hour of benefit.

The second is the framework stack. LangChain plus CrewAI plus AutoGen is not an architecture, it is a shopping list. Four frameworks is four dependency trees, four upgrade paths and four sets of breaking changes to track.

The third is the part most people never price in, so it gets its own section.

The AI agent reliability math nobody runs

AI agents are probabilistic, not deterministic. Given the same input twice they can produce different output, and no amount of prompt guardrailing changes that property. It makes them excellent at judgement and unreliable at repetition.

Now chain them. Say each agent is 95 percent reliable, which is generous. Reliability multiplies along a chain, so the end-to-end number is 0.95 raised to the power of the chain length:

• 3 agents chained: 86 percent. You have already lost 14 points.
• 5 agents chained: 77 percent.
• 10 agents chained: 60 percent. A 40 point drop.

Ten agents is not ten times the capability. It is ten places for the process to fail, and no reliable way to tell which one did it.

Two specifics made it worse in this case. The firm needed to file into DOB NOW and BIS, government portals with no public write API, so submission would have meant automating filings made under a professional engineer's licence. That is legal exposure, not a technical problem. And they wanted a model to set pricing. A model that picks its own prices can invent one. Pricing belongs in a lookup table, not in a prompt.

AI agent cost per job, run properly

Reliability is only half of it. Here is what the two approaches cost to operate, using ballpark figures that hold up at the order-of-magnitude level:

• Agent fleet: roughly 1,400 US dollars a month, and materially more once the same agents run across multiple concurrent jobs.
• Agent fleet, per job: three agents per job, eight model calls each, around 15,000 tokens, so 24 calls at about 1.50 US dollars per job.
• Deterministic build: around 51 US dollars a month. Two LLM calls plus code.
• Deterministic build, per job: about 7 cents.

That is roughly 20 times the running cost per job, for a system that is less accurate and harder to debug. Annualised, the fleet is around 16,800 US dollars a year in model calls alone before anyone has written a line of business logic.

The usual objection is that model costs are falling. They are. But the falling price applies to people vibe-coding an agent with no consequences attached to its output. It does not apply to a system that has to be right about an invoice.

The build hours nobody puts in the quote

Ten agents sharing one security model, one set of guardrails and one test suite is not a weekend project. Realistic ballpark for a full build:

• Core build across ten agents: around 800 hours.
• Audit logging and approval flows: around 150 hours.
• Admin dashboard: around 110 hours.
• Testing, eval harnesses, deployment and documentation: around 90 hours.

You can skip all of it and ship something that demos beautifully. It will not survive contact with a real week of work.

AI agent maintenance cost is what actually decides it

This is the number that ends the argument. With that many moving parts and that many failure points, expect somewhere around 91 hours a month chasing agent failures and keeping eval sets green. That is most of a part-time salary spent babysitting.

The deterministic equivalent runs at roughly six hours a month, because there are fewer dependencies, fewer failure points, and when something breaks the logs tell you exactly which step did it.

Deterministic workflow vs AI agent: what I would build instead

The same ten jobs still get done. The difference is that the AI is used twice, at the two points where judgement is genuinely required, and code does everything else.

1. 1Gmail catches every inbound message.
2. 2One LLM call classifies and extracts. Is this a lead, a supplier invoice, a request for proposal, a change order, a general query? The structured output is plumbed into the CRM.
3. 3A deterministic spine takes over. Every step is logged, every state is inspectable, and the same input always produces the same path.
4. 4An approval gate puts a human in the loop. A quick eyeball on extracted data, not a re-entry of it.
5. 5After approval, a second LLM call selects and runs tools through a single MCP server. QuickBooks, calendar, Postgres, Drive all sit behind one connection layer rather than one agent each.

Two AI steps in the audit log instead of ten. One framework instead of four. When it breaks, and it will, there are two places to look.

The phased roadmap we proposed

Sequencing matters as much as architecture. Each phase pays for the next, and the business feels the benefit in weeks rather than quarters.

1. 1Phase one: email intake, proposal generation and QuickBooks estimates. Entirely deterministic, immediate return, hours freed up straight away.
2. 2Phase two: scheduling, inspection completions and payment reconciliation, built on the same spine.
3. 3Phase three: documents and findings. Pre-fill the regulatory paperwork and leave a human to submit it, so the licensed filing stays with the licensed person.

When an agent fleet is the right call

None of this makes AI agents useless. They earn their place where the task genuinely varies every time, where a wrong answer is cheap to correct, and where a human is reviewing the output anyway. Research, drafting, triage, summarisation, first-pass analysis.

Where they do not earn their place is repeatable back-office process with money or compliance attached. That work wants a state machine with two smart steps in it, not ten smart steps with no state machine.

The honest test before you commission anything: write down the process. If you cannot write it down, an agent will not rescue you, it will inherit the confusion and execute it faster. AI on an undefined process just gets it wrong at speed.

If you are being quoted for an agent fleet right now, run the two numbers in this post against it. Reliability to the power of the chain length, and hours per month to maintain. If those do not survive the arithmetic, the architecture is wrong regardless of how good the demo looked.