Room 42, home

Multi-Agent & Production Agent Architectures

Patterns for orchestrating multiple agents (orchestrator, pipeline, routing and verifier) and running them reliably in production.

RAGadvanced4 min readUpdated 12 Jun 2026

In short

  1. 01Multi-agent systems use roughly fifteen times the tokens of a chat, so reach for them only on high-value, parallelisable work.
  2. 02Common shapes are pipeline, routing, parallelisation, orchestrator and workers, and evaluator and optimiser.
  3. 03Coordination is the hard part: use shared memory, isolate each worker's context, and bound the fan-out.
  4. 04Production needs retries with checkpoints, token budgets, observability, guardrails, and human approval on risky steps.
Contents · 4 sections
  1. 1When is one agent not enough?
  2. 2What are the common multi-agent orchestration patterns?
  3. 3How do multiple agents coordinate and share state?
  4. 4What does it take to run multi-agent systems in production?

One agent in a loop gets you far. Some problems want several agents working together, and a different set of problems only look like they do. This final guide covers the orchestrationThe way you wire several agents together so they hand work off and combine results patterns worth knowing, when more agents help, and what it takes to run them in production.

When is one agent not enough?

More agents is not automatically better. Anthropic's own guidance is blunt about the cost: agents use roughly four times the tokens of a chat, and multi-agent systems around fifteen times. That is only worth it for high-value tasks.

Multiple agents shine on work that is broad and parallelisable, where the information exceeds a single context windowThe fixed amount of text a model can hold in mind at once for a given request, or where there are many tools to juggle. Anthropic reported a multi-agent research system beating a strong single-agent baseline by a wide margin on exactly that kind of open-ended research.

The flip side matters as much. Do not reach for multi-agent when the agents must share the same context or when the subtasks depend tightly on each other. That rules out most coding and most real-time coordination, because models are still not great at delegating to each other on the fly. The rule from Anthropic's "Building Effective Agents" holds: start with the simplest thing that works, and add complexity only when it earns its place.Common gotchaThe classic mistake is splitting a coding task across parallel agents to go faster. The pieces share one evolving codebase, so the workers each edit against a stale picture and stomp each other, and you pay the fifteen-times token bill for a worse, conflicting result.When subtasks touch the same state, keep it in one agent with a longer loop and only fan out work that is independent.

What are the common multi-agent orchestration patterns?

A handful of shapes cover most systems. Each is a different wiring of nodes and edges.

Prompt chaining (pipeline). Decompose a task into ordered steps, with optional checks between them that can fail or branch. Trades latency for accuracy on decomposable work.

Routing. A classifierA small model or step that labels the input so it can be sent to the right handler sends the input to one of several specialists. Each handler can be tuned for its case.

Parallelisation. Split work across workers and combine, either by sectioning one task into independent parts, or by voting (run the same task several times and merge).

Orchestrator and workers. A lead agent breaks the task down at runtime, spawnsCreates and launches a new agent to run a piece of work in its own context workers for the pieces, and synthesises their results. Unlike parallelisation, the work is not known in advance; the orchestrator decides. This is the shape behind most serious research systems, covered in depth in the orchestrator-worker pattern for agents.

Evaluator and optimiser. A generator produces a result, an evaluator critiques it, and the loop repeats until it passes. The same shape covers an adversarial verifier that tries to refute a finding.

How do multiple agents coordinate and share state?

Once you have more than one agent, coordination is the hard part:

  • Shared memory. A common store (a file, a memory store, a database) that agents read and write. Workers leave findings; the lead reads them back.
  • Context isolation. Each worker runs in its own context window and returns only its distilled conclusion, not its whole transcript. That isolation is the point: it lets the system explore more than one window could hold.Common gotchaA frequent error is piping each worker's full transcript back into the lead. The orchestrator's context fills up after a few workers and it loses the thread, which defeats the whole reason you fanned out.Have every worker return a short structured summary instead, and keep its raw steps in shared memory if you need them later.
  • Avoiding runaway loops. Cap iterations, set token and task budgets, and define a clear stop condition. Early multi-agent systems failed by spawning dozens of subagents for trivial queries or searching forever. Bound the fan-out.

What does it take to run multi-agent systems in production?

What separates a demo from a system you can run:

  • Errors compound. Over a long run, one failed step can send an agent down a completely wrong path. Build retries with backoff and durable checkpoints so a failure resumes instead of restarting. Backoff means waiting a little longer before each retryTrying a failed operation again, on the assumption the problem was temporary so you do not hammer a struggling service.

  • Budgets. Enforce per-run token and cost ceilings. The fifteen-times multiplier makes this non-optional, and control AI cost at scale covers the levers in depth.

  • Observability. Agents are non-deterministic between runs, even with the same prompt. You cannot debug what you did not trace, so log every decision and tool call. See add observability to your AI system.

  • Guardrails. Validate tool inputs, and gate hard-to-reverse actions (sends, deletes, external writes) behind confirmation. Keep secrets out of the agent's reachable context. See guardrails for a production AI system.

  • Graceful failure. Surface partial results and the reason a subtask failed, rather than crashing the whole run.

  • Human checkpoints. Put approval gates at the high-stakes, irreversible steps.

The throughline of this whole path holds right to the end: reach for the simplest architecture that does the job, prove it works, and add agents, retrieval, and coordination only where they earn their cost. That is how serious systems get built.