Yiuno

Orchestration

14 min read

Orchestration

Coordinating multiple agents, tools, and decision steps to accomplish a complex task — the conductor that turns individual AI capabilities into a coherent workflow.

What is it?

A single LLM call can answer a question. A pipeline can chain several calls into a sequence. But when the work involves branching decisions, parallel tracks, human checkpoints, error recovery, and multiple agents with different specialities, you need something above the pipeline level — you need orchestration.

Orchestration is the layer that decides what runs, when, in what order, and what happens when something goes wrong.1 It takes the building blocks of pipelines and agents and arranges them into a workflow that can handle real-world complexity: tasks where the plan changes based on intermediate results, where some steps must wait for human approval, and where failure in one branch should not crash the entire system.

The parent concept, agentic-systems, frames orchestration as one of two main components alongside llm-pipelines. Where pipelines concern the internal structure of how work flows through sequential stages, orchestration sits above pipelines and manages the coordination between them — choosing which pipeline to run, running several in parallel, looping back when quality checks fail, and inserting human decision points at critical junctures.2

Anthropic’s research on building effective agents makes a key architectural distinction: workflows are systems where LLMs and tools follow predefined code paths, while agents dynamically direct their own processes.2 Orchestration is what makes both possible at scale. It is the control plane — the system that turns individual capabilities into coordinated action.

In plain terms

A pipeline is one person following a recipe step by step. Orchestration is a head chef running an entire kitchen — deciding which dishes to prepare, assigning cooks to stations, adjusting the plan when an ingredient runs out, and making sure every plate comes out at the right time.

At a glance

How does it work?

Orchestration is built from a set of coordination patterns. Production systems typically combine two or three of these patterns within a single workflow.3

1. The supervisor pattern — centralised control

The most common orchestration pattern uses a single supervisor agent that receives a request, creates an execution plan, delegates tasks to specialist agents, monitors their progress, and assembles the final output.3 The supervisor holds the overall context and makes all routing decisions.

This is Anthropic’s “orchestrator-workers” pattern: a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesises their results. The key difference from a fixed pipeline is flexibility — the subtasks are not predefined but determined by the orchestrator based on the specific input.2

Think of it like...

A film director on set. The director does not operate the camera, act in scenes, or edit the footage. They hold the overall vision, decide what each specialist does, review the results, and adjust the plan when something does not work. The director is the orchestrator; the camera operators, actors, and editors are the specialist agents.

2. Sequential orchestration — ordered stages with gates

The simplest orchestration pattern runs steps in a fixed order, but with decision gates between them that can halt, retry, or reroute the flow.2 This is where orchestration begins to diverge from a basic pipeline: the gates introduce conditional logic that a pure pipeline lacks.

For example: a content publishing workflow might run research, then drafting, then a quality gate. If the gate fails, orchestration sends the draft back for revision rather than pushing a flawed result forward. The pipeline pattern handles the individual stages; orchestration handles the decision logic around them.

Key distinction

A pipeline is a fixed sequence of transformations. Sequential orchestration adds conditional gates, retry logic, and branching — the sequence is the default path, but the orchestrator can deviate from it based on intermediate results.

3. Parallel orchestration — divide and conquer

When subtasks are independent, orchestration can run them simultaneously and aggregate the results.2 This comes in two forms:

  • Sectioning: Different agents handle different aspects of the task concurrently (one checks security, another checks performance, a third checks style)
  • Voting: The same task runs multiple times with different approaches, and the results are compared for consensus

Parallel orchestration requires a synchronisation point — a step where all branches must complete before the workflow continues. The orchestrator manages this synchronisation, handles timeouts for slow branches, and decides what to do if one branch fails while others succeed.3

4. Conditional orchestration — routing and branching

Not every input should follow the same path. A routing step classifies the input and directs it to a specialised handler.2 This is the orchestration pattern that turns a one-size-fits-all system into one that adapts to what it receives.

Routing can be rule-based (keyword matching, metadata checks), model-based (asking an LLM to classify the input), or hybrid. The orchestrator makes the routing decision and then hands off to the appropriate downstream workflow.

Concept to explore

See prompt-routing for a deeper dive into how routing decisions are made and the trade-offs between rule-based and model-based classification.

5. Iterative orchestration — feedback loops

The evaluator-optimiser pattern uses one agent to generate output and another to evaluate it, forming a loop that continues until the output meets quality criteria or a maximum iteration count is reached.2 The orchestrator manages this loop: it decides when to iterate, when to accept, and when to escalate to a human.

This pattern is critical for tasks where quality is measurable but hard to achieve in one pass — translation, code generation, report writing. Each iteration narrows the gap between the current output and the target.4

6. Human-in-the-loop — the human as orchestration checkpoint

Fully autonomous orchestration is appropriate for low-stakes, well-understood tasks. For anything involving irreversible actions, significant cost, or reputational risk, orchestration must include human decision points — moments where the system pauses, presents its work, and waits for approval before continuing.5

Human-in-the-loop is not a failure of automation. It is a deliberate design choice that places human judgement where it matters most while letting agents handle the routine work autonomously.5

Common patterns include:

  • Approval gates: The orchestrator pauses before high-stakes actions (spending money, publishing content, contacting customers) and asks a human to confirm
  • Escalation: The orchestrator routes edge cases or low-confidence decisions to a human rather than guessing
  • Review checkpoints: The orchestrator presents intermediate results at defined intervals for human quality checks

Think of it like...

A surgical team. The surgeon (human) makes the critical decisions — where to cut, when to stop. But the instruments are prepared, the patient is monitored, and routine steps are handled by the team (agents) autonomously. The surgeon does not do everything; they intervene at the moments that matter. Good orchestration puts the human in the surgeon’s role, not the nurse’s.

Concept to explore

See human-in-the-loop for a deeper dive into when and how to insert human decision points in agentic workflows.

7. Error handling and recovery

Orchestration is where error handling lives. Individual agents may fail — an API times out, a model produces invalid output, a tool returns an error. The orchestrator’s job is to detect failures and decide what to do: retry the step, try an alternative approach, skip the step and note the gap, or escalate to a human.3

Good orchestration design treats failure as a normal part of the workflow, not an exception. Production systems include:

  • Retry policies: Automatic retries with exponential backoff for transient failures
  • Fallback paths: Alternative agents or models that activate when the primary option fails
  • Graceful degradation: Producing a partial result with noted gaps rather than failing entirely
  • Circuit breakers: Stopping repeated calls to a failing service to avoid cascading failures

Key distinction

A pipeline fails when any stage fails — the chain breaks. An orchestrator can route around failures, retry with different parameters, or produce a partial result. This resilience is one of the main reasons to use orchestration over a simple pipeline for production systems.

Why do we use it?

Key reasons

1. Complex tasks need coordination, not just sequence. Real-world work involves branching, parallelism, and human decisions that a linear pipeline cannot express. Orchestration provides the control structures — conditionals, loops, parallel execution, human gates — that complex workflows require.2

2. Failure resilience. In production, things break. APIs time out, models hallucinate, tools return errors. Orchestration provides retry logic, fallback paths, and graceful degradation so that a single failure does not collapse the entire workflow.3

3. Human-AI collaboration. Pure automation is appropriate for some tasks, but many require human judgement at critical points. Orchestration makes it possible to combine the speed of agents with the judgement of humans, placing each where they add the most value.5

4. Observability and control. An orchestrator provides a single point from which to monitor the entire workflow — what is running, what has completed, what has failed. This makes debugging, auditing, and improving the system far easier than tracing through independent agents.3

When do we use it?

  • When the task involves multiple agents or pipelines that need to work together
  • When the execution path depends on intermediate results — not every input follows the same steps
  • When some steps require human approval before proceeding
  • When failure in one part should not crash the entire workflow
  • When you need to run steps in parallel and synchronise the results
  • When the workflow requires iterative refinement — generating, evaluating, and improving in a loop

Rule of thumb

If your task can be described as a straight line of steps, a pipeline is enough. If you need branching, looping, parallel tracks, human checkpoints, or error recovery, you need orchestration.

How can I think about it?

The air traffic controller

An air traffic controller manages dozens of aircraft, each following its own flight plan, through a shared airspace.

  • Each aircraft = an agent or pipeline executing its own task
  • The flight plan = the instructions each agent follows
  • The controller = the orchestrator, holding the big picture
  • Sequencing takeoffs and landings = sequential orchestration with gates
  • Multiple aircraft in holding patterns = parallel execution waiting for synchronisation
  • Rerouting around a storm = conditional branching when circumstances change
  • Handing off to the pilot for landing = human-in-the-loop at a critical moment
  • Declaring an emergency priority = error handling and escalation

The controller does not fly any aircraft. They coordinate all of them, resolve conflicts, and ensure the overall system works safely. Orchestration does the same for AI agents.

The wedding planner

A wedding planner coordinates dozens of vendors — caterer, florist, musician, photographer, venue — to deliver a single coherent event.

  • Each vendor = a specialist agent with its own tools and expertise
  • The wedding planner = the orchestrator
  • The timeline = the execution plan (ceremony, then photos, then reception)
  • Parallel preparation = caterer and florist working simultaneously
  • Decision gates = checking that the cake arrived before the reception starts
  • Plan B for rain = conditional branching and error recovery
  • The couple approving the seating chart = human-in-the-loop checkpoint

No single vendor could run the entire wedding. The value is in coordination — making sure the right people do the right things at the right time, and adjusting when the florist is late or the weather changes. That is orchestration.

Concepts to explore next

ConceptWhat it coversStatus
playbooks-as-programsHow structured playbooks serve as executable orchestration plans for agentsstub
human-in-the-loopWhen and how to insert human decision points in agentic workflowsstub

Some cards don't exist yet

A broken link is a placeholder for future learning, not an error.

Check your understanding

Where this concept fits

Position in the knowledge graph 

graph TD
    AS[Agentic Systems] --> LP[LLM Pipelines]
    AS --> ORCH[Orchestration]
    LP -.->|building block for| ORCH
    ORCH --> PP[Playbooks as Programs]
    ORCH --> HITL[Human-in-the-Loop]
    style ORCH fill:#4a9ede,color:#fff

Related concepts:

  • llm-pipelines — pipelines are the building blocks that orchestration coordinates; a pipeline is one step, orchestration manages many
  • context-cascading — orchestration relies on layered context to ensure each agent receives the right information for its specific task
  • prompt-routing — routing is one of the core orchestration patterns, classifying input and directing it to specialised handlers
  • knowledge-graphs — structured knowledge can inform orchestration decisions, helping the orchestrator choose the right plan based on what is known

Sources

Further reading

Resources

Footnotes

  1. The Thinking Company. (2026). AI Agent Orchestration Patterns (2026 Guide). The Thinking Company. ↩

  2. Schluntz, E. and Zhang, B. (2024). Building Effective Agents. Anthropic. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8

  3. Nerd Level Tech. (2026). Mastering Agent Orchestration Patterns: From Theory to Production. Nerd Level Tech. ↩ ↩2 ↩3 ↩4 ↩5 ↩6

  4. Carpintero, D. (2025). Design Patterns for Building Agentic Workflows. Hugging Face. ↩

  5. Renner, K. (2026). Human-in-the-Loop Agent Patterns: When Agents Should Ask for Help. Harness Engineering Academy. ↩ ↩2 ↩3

Graph View

Backlinks