Anthropic's Orchestrator-Worker Pattern for Variable Task Decomposition
TRIGGER
Some tasks can't be decomposed into fixed subtasks ahead of time—the number and nature of subtasks depends on the specific input. Hardcoding parallel paths fails because you can't predict what work is needed until you analyze the request.
APPROACH
A central orchestrator LLM analyzes the task and dynamically breaks it into subtasks, delegates each to worker LLMs, then synthesizes their results. Input: complex task description. Output: coordinated result from multiple workers. Used in coding products where changes span multiple files—the orchestrator determines which files need changes and what kind of change each needs based on the specific task, then dispatches workers accordingly.
PATTERN
“One task touches 1 file, another touches 20, and you cannot predict which until you see the input. Let an orchestrator LLM decompose at runtime; workers execute. Dynamic structure beats hardcoded branches that cannot anticipate variance.”
✓ WORKS WHEN
- Subtask count and nature vary significantly across inputs (coding: 1 file vs 20 files depending on task)
- Subtasks are independent enough to delegate to separate workers
- Orchestrator can reliably understand task structure from input alone
- Worker results can be synthesized into coherent output
✗ FAILS WHEN
- Subtask structure is predictable and fixed across all inputs (use parallelization instead)
- Subtasks have complex interdependencies requiring tight coordination
- Overhead of orchestration exceeds benefit of parallel workers
- Task is simple enough that single LLM call handles it without decomposition