Skip to content

Workflow vs Agent#

The workflow vs agent LLM decision is the most consequential architectural choice in an LLM product. A workflow is a fixed DAG: deterministic steps, each with one or more LLM calls. An agent is a loop: the LLM decides what to do next on every turn.

flowchart LR
  subgraph WF[Workflow: fixed DAG]
    A1[Step 1<br/>extract]
    A1 --> A2[Step 2<br/>classify]
    A2 --> A3[Step 3<br/>route]
    A3 --> A4[Step 4<br/>respond]
  end
  subgraph AG[Agent: autonomous loop]
    B1[Think] --> B2[Act<br/>pick tool]
    B2 --> B3[Observe]
    B3 --> B1
    B1 --> B4[Final]
  end

    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    class A1,A2,A3,A4 service;
    class B1,B2,B3,B4 compute;

Workflows win when the problem decomposes cleanly into known steps: extract claims from emails, classify each, route to a queue, draft a reply. You get predictable cost, low latency, easy debugging, and trivial rollback.

Agents win when the path is unknown: research a topic across the web, debug a failing test, navigate a UI. You give up predictability for flexibility. Cost can balloon 5-20x because the model decides how many turns to take.

The hybrid pattern is the production default: the agent runs inside guardrails set by a workflow. The outer workflow handles auth, rate limits, output format, and termination; the inner agent loop handles the actual reasoning.

Decision factors: predictability (workflow wins), debuggability (workflow), cost (workflow), recovery from new failure modes (agent), open-ended problem space (agent), tools that change (agent). When in doubt, ship a workflow first; promote to agent only when its rigidity blocks users.

Problem statement

Your team is building an AI feature: read customer support emails, find the relevant order, write a draft reply, send for human review. A junior engineer proposes "let's give the LLM tools and let it figure it out." A senior engineer says "no, it should be a five-step pipeline." Both are partially right. How do you decide workflow vs agent LLM and design the hybrid?

This is the most-debated architectural choice in applied LLM work. Anthropic's "Building Effective Agents" blog post (Dec 2024) settled the language: a workflow is an orchestrated path of LLM calls through predefined code; an agent is a system where the LLM dynamically directs its own process and tool use.

The spectrum#

flowchart LR
  P[Pure prompt] --> R[Router] --> W[Workflow DAG] --> WT[Workflow + tools] --> AG[Agent w/ guardrails] --> FA[Free agent]

    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    class P compute;
    class R,W,WT service;
    class AG,FA datastore;

Real systems sit along a spectrum:

Level Description Example
Pure prompt One LLM call, no tools Summarize a paragraph
Router LLM picks one branch from N Classify intent, dispatch
Workflow DAG Fixed sequence of LLM steps Extract -> classify -> compose
Workflow + tools DAG nodes can call tools Step 3 calls a DB query
Agent with guardrails Loop inside a workflow shell Outer auth + budget, inner ReAct
Free agent Open-ended loop "Plan my trip" with all tools

Decision factors#

Factor Workflow Agent
Predictability High, fixed path Low, model-driven
Debuggability One step, one bug Loop with branching state
Cost Bounded, often <5 calls Unbounded, can hit 50+
Latency Deterministic Variable, often 10-30s
Recovery Limited to coded retries Model can self-correct
Open-endedness Poor, only what's in DAG Strong
Tool churn Code change per new tool Add to registry, no code
Onboarding new dev Easy, read DAG Hard, read traces
Auditability Strong, fixed flow Weak, per-instance trace

A rule that holds 80 percent of the time: if you can write the DAG on a whiteboard in under 5 minutes, build a workflow.

Workflow patterns (Anthropic taxonomy)#

flowchart TB
  subgraph PC[Prompt chaining]
    P1[Step 1: extract] --> P2[Step 2: refine] --> P3[Step 3: render]
  end
  subgraph RT[Routing]
    R1[Classifier] --> RA[Branch A]
    R1 --> RB[Branch B]
    R1 --> RC[Branch C]
  end
  subgraph PR[Parallelization]
    PP1[Aspect 1]
    PP2[Aspect 2]
    PP3[Aspect 3]
    PP1 --> AG2[Aggregator]
    PP2 --> AG2
    PP3 --> AG2
  end
  subgraph OR[Orchestrator-workers]
    OO[Orchestrator LLM] --> WK1[Worker 1]
    OO --> WK2[Worker 2]
    WK1 --> OO
    WK2 --> OO
  end
  subgraph EV[Evaluator-optimizer]
    EE[Generator] --> EJ[Judge]
    EJ -->|fix| EE
    EJ -->|ok| EX[Out]
  end

    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    class P1,P2,P3,R1,RA,RB,RC,OO,WK1,WK2,EE,EJ,EX service;
    class PP1,PP2,PP3,AG2 compute;

Five workflow patterns cover most non-agent cases:

  1. Prompt chaining: each LLM output feeds the next. Validate at each step.
  2. Routing: classifier picks a downstream branch.
  3. Parallelization: split, run independently, aggregate. Great for "analyze N facets".
  4. Orchestrator-workers: a planner LLM decomposes the task into subtasks dispatched to worker LLMs.
  5. Evaluator-optimizer: generator + judge loop until quality threshold.

Note that orchestrator-workers and evaluator-optimizer have a structural loop, but the loop is bounded and the control logic is in your code, not the LLM.

When to lean agent#

Agents shine when:

  • The decision tree is too wide to enumerate (research, troubleshooting)
  • Tools are added at runtime, not at code time (MCP servers)
  • The user gives an open-ended goal ("plan my trip", "fix this bug")
  • Recovery from novel failures matters more than peak efficiency

Examples that genuinely need an agent:

  • Cursor / Claude Code: arbitrary codebase exploration
  • Computer use: GUI navigation
  • Deep research: multi-hop web search with synthesis
  • AI SRE: investigate an alert by querying logs, metrics, and traces

Hybrid: agent-with-guardrail-DAG#

The pragmatic production architecture:

flowchart TB
  R([Request]) --> AUTH[Outer: auth + tenant scope]
  AUTH --> BUDGET[Set token + step budget]
  BUDGET --> AGENT[Inner: ReAct loop<br/>tools, memory, retries]
  AGENT --> CHECK{Output valid?}
  CHECK -- no --> AGENT
  CHECK -- yes --> POST[Outer: format, redact, log]
  POST --> SHIP[Send to user]

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    class R client;
    class AUTH,BUDGET,POST,SHIP service;
    class AGENT,CHECK compute;

The outer workflow owns:

  • Authentication, tenant scoping, rate limiting
  • Token, step, and dollar budgets
  • Required tool denylist (e.g. "no send_email without human confirm")
  • Final output schema validation
  • Audit logging and redaction

The inner agent owns:

  • Tool selection
  • Reasoning trace
  • Retry on tool errors

You get predictability where you need it (compliance, cost) and flexibility where you need it (reasoning).

Cost math#

For a customer-support draft reply task, comparing pure workflow vs free agent vs hybrid:

Architecture Avg LLM calls Avg tokens Cost / request Time
Workflow (5 steps) 5 8K $0.06 3s
Free agent 12 (range 4-30) 25K $0.18 18s
Hybrid (capped at 6 steps) 6 12K $0.08 6s

Free agents are 3x more expensive on average and have a fat tail. A user who asks something complex can trigger a $5 trace. Step budgets are non-optional in production.

Worked examples#

Pure workflow: support email triage#

def triage(email):
    # Step 1: extract
    extracted = llm("Extract: order_id, issue_type, urgency", email)
    # Step 2: lookup
    order = db.get_order(extracted.order_id)
    # Step 3: classify
    intent = llm("Classify into [refund, shipping, defect, other]", extracted, order)
    # Step 4: route
    if intent == "refund" and order.value > 500:
        return "human_review"
    # Step 5: draft
    draft = llm("Write reply", template_for(intent), extracted, order)
    return draft

Predictable, fast, cheap. Adding a new intent type = one branch.

Free agent: deep research#

def research(question):
    return agent_loop(
        question,
        tools=[web_search, web_fetch, summarize, citations],
        max_steps=20,
        max_tokens=100_000,
    )

The model decides how many searches, what to fetch, when to synthesize. Acceptable cost variance because the task is high-value.

Hybrid: AI SRE incident triage#

def triage_alert(alert):
    # Outer: pre-flight
    assert tenant_allowlist(alert.tenant_id)
    budget = Budget(steps=8, tokens=50_000, dollars=0.50)

    # Inner: agent investigates
    findings = agent_loop(
        goal=f"Diagnose {alert}",
        tools=[query_logs, query_metrics, get_trace, list_deploys],
        budget=budget,
    )

    # Outer: format and notify
    validate_schema(findings, IncidentSchema)
    return notify_oncall(findings)

Failure modes per pattern#

Pattern Failure Mitigation
Workflow New case falls outside DAG Add escape valve: route to agent or human
Workflow Step error stalls pipeline Per-step retry + DLQ
Free agent Infinite tool loop Step + budget cap, duplicate-call detector
Free agent Wrong tool selected Tighten descriptions, add few-shot examples
Hybrid Outer rejects valid agent output Loosen schema or add repair step
Hybrid Agent ignores guardrail tools Strip disallowed tools from registry per request

Progression strategy#

The strategy that wins for most teams:

  1. Ship a workflow in week 1
  2. Instrument it: where do users escape outside the DAG?
  3. For those escape cases, add an "ask agent" fallback branch
  4. As agent fallback succeeds, expand its tool set
  5. Eventually the workflow shell stays, the inner gets more agent

Anthropic's guidance: most production LLM features are workflows, not agents. Reserve full agent freedom for genuinely open-ended user goals.

Quick reference#

Definitions (Anthropic)#

  • Workflow: orchestrated path of LLM calls through predefined code
  • Agent: LLM dynamically directs its own process and tool use

Spectrum#

  1. Pure prompt
  2. Router (LLM picks branch)
  3. Workflow DAG
  4. Workflow + tools
  5. Agent with guardrails (hybrid)
  6. Free agent

Decision table#

Factor Workflow Agent
Predictability High Low
Debuggability Easy Trace-heavy
Cost Bounded Unbounded
Latency Deterministic Variable
Recovery Coded retries Self-correct
Open-ended Poor Strong
Tool churn Code change Registry add

Whiteboard rule: DAG draws in 5 minutes -> workflow.

Five workflow patterns#

  1. Prompt chaining (extract -> refine -> render)
  2. Routing (classify, branch)
  3. Parallelization (split, aggregate)
  4. Orchestrator-workers (planner LLM dispatches)
  5. Evaluator-optimizer (generator + judge loop)

When agent wins#

  • Decision tree too wide to enumerate
  • Tools added at runtime (MCP)
  • Open-ended user goals
  • Novel-failure recovery > peak efficiency

Hybrid (production default)#

Outer workflow: - Auth, tenant scope - Token / step / dollar budget - Tool allowlist - Schema validation - Audit log

Inner agent: - Tool selection - Reasoning - Retry on errors

Cost example (support triage)#

Arch Calls Tokens $/req Time
Workflow 5-step 5 8K $0.06 3s
Free agent 12 25K $0.18 18s
Hybrid capped 6 12K $0.08 6s

Free agents have a fat tail: a single hard request can cost $5+. Budgets are non-optional.

Failure modes#

Pattern Failure Fix
Workflow Case outside DAG Escape to agent/human
Workflow Step stalls Per-step retry + DLQ
Agent Loop forever Step + budget cap
Agent Wrong tool Tighten descriptions
Hybrid Schema rejects valid out Add repair step

Progression strategy#

  1. Ship workflow week 1
  2. Instrument: where do users escape DAG?
  3. Add agent fallback branch for those
  4. Grow agent tools as fallback succeeds
  5. Outer shell stays, inner gets more agent

Anthropic guidance#

Most production LLM features are workflows, not agents. Reserve full agent freedom for genuinely open-ended user goals (Cursor, Claude Code, deep research, AI SRE).

Refs#

  • Anthropic, Building Effective Agents (Dec 2024)
  • Yao et al., ReAct (2022)
  • DSPy, LangGraph, Temporal AI patterns
  • LlamaIndex Workflows docs

FAQ#

What is the difference between a workflow and an agent in LLM apps?#

A workflow is a fixed DAG of deterministic steps with one or more LLM calls per step. An agent is a loop where the LLM decides the next action every turn, including which tool to call.

When should I use a workflow instead of an agent?#

Use a workflow when the steps are known up front, latency matters, and cost or reliability are critical. Workflows are easier to test, observe, and debug than autonomous agents.

When does an agent beat a workflow?#

Agents win when the path through the steps depends on observations only known at runtime. Open-ended research, code repair, and complex troubleshooting are natural fits for an agent loop.

Are agents more expensive than workflows?#

Usually yes. Agents iterate with tool calls and grow context as they go, which can multiply token usage by 5 to 20 times compared to a fixed workflow that runs the same steps once.

Can you mix workflows and agents?#

Yes, and most production systems do. A workflow handles the predictable scaffolding while an agent loop is scoped inside one node to handle the unpredictable middle of the problem.