Multi-Agent Research

A multi-agent research system design uses a supervisor LLM to break a hard question into smaller searches, dispatches those searches to parallel sub-agents, and stitches their findings into a single cited report. Anthropic's research mode and OpenAI's deep research follow this pattern: one model plans, many models search, and one model writes.

Problem statement (interviewer prompt)

Design a research assistant that, given an open-ended question, spawns multiple parallel sub-agents to gather evidence from the web and internal documents, then synthesizes a cited report. The system should finish in 1 to 5 minutes for typical questions, handle 100 concurrent research sessions, and stay under $1 per session in LLM and search API spend.

flowchart TB
  U([User])
  GW[API Gateway]
  SUP[Supervisor agent<br/>plans, decomposes]
  SCRATCH[(Shared scratchpad<br/>memory store)]
  SUB1[Sub-agent 1<br/>topic A search]
  SUB2[Sub-agent 2<br/>topic B search]
  SUB3[Sub-agent 3<br/>topic C search]
  TOOLS[Search and fetch tools<br/>web, RAG, code]
  SYN[Synthesizer<br/>writes final report]
  STORE[(Report store)]

  U --> GW
  GW --> SUP
  SUP --> SCRATCH
  SUP --> SUB1
  SUP --> SUB2
  SUP --> SUB3
  SUB1 --> TOOLS
  SUB2 --> TOOLS
  SUB3 --> TOOLS
  SUB1 --> SCRATCH
  SUB2 --> SCRATCH
  SUB3 --> SCRATCH
  SCRATCH --> SYN
  SYN --> STORE
  STORE --> U

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    class U client;
    class GW edge;
    class SUP,SYN service;
    class SUB1,SUB2,SUB3 compute;
    class SCRATCH,STORE datastore;
    class TOOLS external;

The supervisor is the brain. It reads the question, drafts a plan, and decides which sub-questions are worth a sub-agent and which it can answer itself. The cost of spawning a sub-agent is real, both in tokens and in coordination overhead, so the supervisor only fans out when a sub-task is meaty enough to fill a few search turns. Trivial lookups stay in the main loop.

Sub-agents are isolated workers. Each one gets a single sub-question, an empty conversation, a set of search and fetch tools, and a token budget. They explore independently, accumulate citations, and emit a structured report block. Isolation matters: if a sub-agent gets stuck in a loop, it only burns its own budget, not the whole session.

The shared scratchpad is the glue. Sub-agents write their findings, along with source URLs and snippet IDs, to a session-scoped store. The synthesis stage reads the scratchpad in full, deduplicates evidence, resolves contradictions, and writes the final cited report. Citations carry forward as inline references that link back to the original source so the user can verify every claim.

Problem statement (interviewer prompt)

Design a multi-agent research system in the style of Anthropic's research mode or OpenAI deep research. Given an open-ended question, the system decomposes it into sub-questions, spawns parallel sub-agents that search the web and internal sources, accumulates evidence in a shared memory, and synthesizes a cited final report. Targets: 1 to 5 minute end-to-end latency, 100 concurrent sessions, average cost under $1/session, and every factual claim cited with a verifiable source.

Requirements

Functional

• Accept a free-form research question, optionally with attached files.
• Plan: produce a sub-question decomposition, possibly multi-level.
• Spawn N sub-agents in parallel, each with their own scratch context.
• Tools: web_search, fetch_url, rag_search (internal corpus), code_exec, read_file.
• Stream progress updates (which sub-agent is searching what).
• Produce a final markdown report with inline citations.
• Allow user follow-ups that reuse the prior session's evidence.

Non-functional

• p50 end-to-end latency 2 minutes, p95 5 minutes.
• 100 concurrent sessions, 10k sessions/day peak.
• Average cost per session under $1.
• 99% citation accuracy: cited claims must be findable in the source.
• Graceful degradation: partial results if one sub-agent times out.

Top-level architecture

flowchart TB
  subgraph Client[Clients]
    UI([Web UI])
    API([API consumer])
  end

  subgraph Edge[Edge]
    GW[Gateway<br/>auth, rate limit]
    STREAM[SSE stream<br/>progress events]
  end

  subgraph Plane[Control plane]
    SCHED[Session scheduler<br/>concurrency caps]
    SUP[Supervisor runner<br/>planner + reducer]
    POOL[Sub-agent worker pool<br/>autoscaling]
  end

  subgraph Mem[Session memory]
    SCRATCH[(Scratchpad<br/>Redis + Postgres)]
    CITE[(Citation graph<br/>Postgres)]
    BLOB[(Snippet blobs<br/>S3)]
  end

  subgraph Tools[Tool services]
    WS[Web search<br/>Brave / Bing / Tavily]
    FETCH[Fetch + reader<br/>HTTP + readability]
    RAG[Internal RAG<br/>vector DB]
    CODE[Code exec sandbox]
  end

  subgraph LLMs[Model providers]
    BIG[Frontier model<br/>Sonnet / GPT-4]
    SMALL[Cheap model<br/>Haiku, for re-rank]
  end

  subgraph Data[Outputs]
    REP[(Report store<br/>Postgres)]
    EVT[(Event log<br/>ClickHouse)]
  end

  UI --> GW
  API --> GW
  GW --> SCHED
  SCHED --> SUP
  SUP --> POOL
  SUP --> SCRATCH
  POOL --> SCRATCH
  POOL --> CITE
  POOL --> BLOB
  POOL --> WS
  POOL --> FETCH
  POOL --> RAG
  POOL --> CODE
  POOL --> BIG
  POOL --> SMALL
  SUP --> BIG
  SUP --> REP
  GW --> STREAM
  SUP -.events.-> EVT
  STREAM --> UI

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    class UI,API client;
    class GW,STREAM edge;
    class SCHED,SUP,POOL service;
    class SCRATCH cache;
    class CITE,REP,EVT datastore;
    class BLOB storage;
    class WS,FETCH,RAG,CODE,BIG,SMALL external;

Data model

sessions(
  id uuid PK,
  user_id uuid,
  question text,
  status text,             -- planning | researching | synthesizing | done | failed
  plan jsonb,              -- supervisor's task tree
  started_at timestamptz,
  finished_at timestamptz,
  cost_micros bigint,
  tokens_total int
);

sub_tasks(
  id uuid PK,
  session_id uuid FK,
  parent_id uuid,          -- null for root
  question text,
  assigned_to text,        -- self | sub_agent
  status text,             -- pending | running | done | timeout | failed
  iterations int,
  tokens_used int,
  result_blob_id uuid
);

citations(
  id uuid PK,
  session_id uuid FK,
  source_url text,
  source_title text,
  snippet text,
  snippet_hash text,
  fetched_at timestamptz,
  used_in_report bool
);

claim_to_citation(
  claim_id uuid,
  citation_id uuid,
  PRIMARY KEY (claim_id, citation_id)
);

reports(
  id uuid PK,
  session_id uuid FK,
  markdown text,
  outline jsonb,
  citation_ids uuid[],
  created_at timestamptz
);

Component deep-dive

Supervisor: plan and dispatch

The supervisor runs in a single LLM context. Its job is to read the question, ask one or two clarifying turns (or skip if the question is concrete), and emit a plan: a list of sub-questions with a recommended approach for each.

flowchart LR
  Q[User question]
  CLAR{Needs clarification?}
  ASK[Ask user]
  PLAN[Draft plan<br/>3 to 8 sub-questions]
  REVIEW[Self-review pass]
  DISP[Dispatch sub-agents]
  WAIT[Wait or stream partials]
  REDUCE[Reduce results]
  SYN[Final synthesis]

  Q --> CLAR
  CLAR -- yes --> ASK
  ASK --> PLAN
  CLAR -- no --> PLAN
  PLAN --> REVIEW
  REVIEW --> DISP
  DISP --> WAIT
  WAIT --> REDUCE
  REDUCE --> SYN

    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    class Q,SYN service;
    class CLAR,ASK,PLAN,REVIEW,DISP,WAIT,REDUCE compute;

The "spawn vs do-yourself" heuristic is the most important supervisor decision. Cheap rules of thumb that work in practice:

• Sub-questions that need more than two searches each are worth a sub-agent.
• Sub-questions that need different domains (legal vs technical, US vs EU) parallelize well.
• Pure fact lookups ("What is the capital of...") are not worth spawning.
• A second-level fanout is rarely worth it. Two levels are an interview red flag without strong evidence.

Sub-agent worker pool

Each sub-agent is a fresh LLM context with its own ReAct loop. The pool runs as a horizontally autoscaled worker fleet pulling tasks from an in-memory queue scoped to the session.

sequenceDiagram
  participant SUP as Supervisor
  participant Q as Session queue
  participant W as Worker (sub-agent)
  participant T as Tool service
  participant M as Scratchpad

  SUP->>Q: enqueue 4 sub-tasks
  par
    Q-->>W: task A
    W->>T: web_search("...")
    T-->>W: top 10 results
    W->>T: fetch_url(top 3)
    T-->>W: cleaned text
    W->>M: write findings + citations
    W-->>SUP: task A done
  and
    Q-->>W: task B
    W->>T: rag_search
    T-->>W: chunks
    W->>M: write findings
    W-->>SUP: task B done
  end
  SUP->>M: read all findings
  SUP->>SUP: synthesize

Each sub-agent has hard limits: 12 tool calls, 50k total tokens, 90 second wall clock. Hitting any limit produces a partial result that the supervisor can still incorporate, with a flag noting the truncation. This prevents one runaway sub-agent from sinking the whole session.

Shared scratchpad and citation graph

The scratchpad is structured, not freeform. Each entry has a sub-task ID, a finding (1 to 3 sentences), and a list of citation IDs. Citations live in their own table because the same source often supports multiple findings, and we want to deduplicate them in the final reference list.

flowchart LR
  S1[Sub-agent A] --> F1[Finding<br/>+ cite refs]
  S2[Sub-agent B] --> F2[Finding<br/>+ cite refs]
  S3[Sub-agent C] --> F3[Finding<br/>+ cite refs]
  F1 --> SP[(Scratchpad)]
  F2 --> SP
  F3 --> SP
  SP --> CG[(Citation graph<br/>dedup by URL hash)]
  CG --> SYN[Synthesizer reads<br/>findings + citations]
  SYN --> R["Final report<br/>with [1][2] refs"]

    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    class S1,S2,S3,SYN compute;
    class SP,CG datastore;
    class F1,F2,F3,R service;

The citation graph also enables verification. After the report is drafted, a verifier pass takes each claim, looks up its cited snippets, and re-prompts a cheap model with "Does this snippet support this claim?". Claims that fail verification get flagged or rewritten.

Synthesis stage

Synthesis is a separate LLM call that gets the full scratchpad (often 30 to 60k tokens of findings) plus the original question and outline. It writes the report in sections, each section drawing from a specific set of findings. Inline citations use a [claim_id] marker that is post-processed into numbered footnotes pointing to the citation table.

If the scratchpad is too large to fit, a map-reduce pre-pass summarizes each sub-agent's findings down to 2k tokens before synthesis.

Sample code: supervisor loop sketch

from dataclasses import dataclass
import asyncio

@dataclass
class SubTask:
    id: str
    question: str
    budget_tokens: int = 50_000
    budget_seconds: int = 90

async def run_research(question: str, scratchpad, citations, model) -> str:
    plan = await model.complete(
        system=SUPERVISOR_PLANNER_PROMPT,
        user=question,
        response_format="json",
    )
    sub_tasks = [SubTask(id=str(i), question=q) for i, q in enumerate(plan["subqs"])]

    async def run_sub(t: SubTask):
        try:
            return await asyncio.wait_for(
                _sub_agent(t, scratchpad, citations),
                timeout=t.budget_seconds,
            )
        except asyncio.TimeoutError:
            return {"task_id": t.id, "status": "timeout", "partial": True}

    results = await asyncio.gather(*(run_sub(t) for t in sub_tasks))

    findings = scratchpad.read_all()
    citations_list = citations.dedupe()
    report = await model.complete(
        system=SYNTHESIZER_PROMPT,
        user={
            "question": question,
            "findings": findings,
            "citations": citations_list,
            "outline_hint": plan["outline"],
        },
    )
    return _attach_footnotes(report, citations_list)

Failure modes and scaling

Failure	Effect	Mitigation
Sub-agent runs forever	Session stalls, cost spike	Token + wall-clock budgets, partial results allowed
All sub-agents return weak evidence	Low-quality report	Supervisor re-plans, optionally spawns a second wave
Web search rate-limited	Tools 429	Multiple providers behind a router, exponential backoff
Citation does not support claim	Hallucination in output	Post-synthesis verifier pass with cheap model
Scratchpad too large for synthesis	Synthesis fails	Map-reduce summarize per sub-agent, fall back to outline-driven write
Provider outage mid-session	Half-finished session	Checkpoint after each sub-agent, resume with alternate provider

Scale-out: the worker pool is horizontally autoscaled on queue depth. A single supervisor can manage many concurrent sessions because most of its time is spent waiting on sub-agents. Per-session concurrency caps (default: 5 sub-agents in parallel) prevent one user from monopolizing the search API quota. Sessions are stateful (the scratchpad) but the supervisor is restartable: the scratchpad + sub-task table is the durable source of truth, so a crashed supervisor pod resumes by re-reading state.

Cost considerations

Assume average session: 1 supervisor pass (10k in, 2k out), 5 sub-agents averaging 30k input and 5k output each, 1 synthesis call (40k in, 4k out), 1 verifier pass (20k in, 1k out).

• Supervisor: 10k * $3/1M + 2k * $15/1M = $0.06
• Sub-agents: 5 * (30k * $3/1M + 5k * $15/1M) = 5 * $0.165 = $0.825
• Synthesis: 40k * $3/1M + 4k * $15/1M = $0.18
• Verifier (cheap model): 20k * $0.25/1M + 1k * $1.25/1M = $0.0065
• Search and fetch APIs: ~$0.02/session
• Total: ~$1.09/session before optimization.

With prompt caching on the synthesis stage and cheaper sub-agent models (Sonnet vs Opus) the realistic average lands near $0.70/session. The largest cost is sub-agent token consumption, so tightening sub-agent budgets is the highest-leverage knob.

Latency budget

Target: p50 end-to-end 2 minutes.

Step	Budget	Notes
Plan (supervisor turn 1)	10 s	One LLM call
Sub-agents (parallel, longest path)	60 s	4 to 6 tool calls each, ~10 s per call
Reduce + handoff	5 s	Read scratchpad, format
Synthesis	30 s	Large input, streaming output
Verifier	10 s	Cheap model, parallel per claim
Stream final to UI	5 s	Markdown render
Total p50	~2 min

The critical path is the slowest sub-agent. The system streams partial findings to the UI so users see progress; they almost never wait passively for 2 minutes without feedback.

Quick reference

Functional requirements

• Accept open-ended question (optionally with files).
• Supervisor produces a sub-question plan.
• Spawn N parallel sub-agents with isolated contexts.
• Tools: web_search, fetch_url, rag_search, code_exec, read_file.
• Stream progress events to the UI.
• Produce a markdown report with inline citations.
• Support follow-up questions reusing prior evidence.

Non-functional

• p50 latency 2 minutes, p95 5 minutes.
• 100 concurrent sessions, 10k/day peak.
• Cost target under $1/session average.
• 99% citation accuracy.

Capacity (back-of-envelope)

• 10k sessions/day, 5 sub-agents/session = 50k sub-agent runs/day.
• ~$0.70/session at steady state = $5k/day LLM spend.
• Web search API: ~$0.005/query, ~5 queries/sub-agent = $1.25k/day.
• Scratchpad: ~50 KB/session, all sessions = ~500 MB/day Postgres.

Key services

• Gateway: auth, rate limit, SSE for progress events.
• Session scheduler: caps concurrent sub-agents per tenant.
• Supervisor: plan, dispatch, reduce, synthesize.
• Worker pool: autoscaled sub-agent runners.
• Scratchpad (Redis active + Postgres durable): structured findings.
• Citation graph (Postgres): dedup by URL hash, link claims to sources.
• Tool services: web search, URL fetch + reader, internal RAG, code exec.
• Verifier: cheap model post-pass to check claims against citations.

When to spawn a sub-agent

• Sub-question needs more than 2 searches.
• Sub-question covers a different domain or source set.
• Multiple sub-questions can run in true parallel.

When NOT to spawn

• Single fact lookup.
• Question already answered by supervisor's training knowledge.
• Total session token budget is nearly exhausted.
• Second-level fanout: usually wasteful, rare exceptions.

Sub-agent limits (defaults)

• 12 tool calls max.
• 50k total tokens.
• 90 second wall-clock.
• Hitting any limit yields a partial result, not a hard failure.

Common pitfalls

• Letting sub-agents run unbounded; one bad task burns the whole session.
• No verifier pass; hallucinated citations slip through.
• Storing scratchpad as freeform text instead of structured findings + citation refs.
• Synthesis with too-large context; map-reduce first.
• Single search provider; rate limits stall sessions.
• Streaming the final report only after synthesis instead of streaming intermediate progress.
• Treating supervisor and sub-agent as the same prompt; they have different jobs and budgets.

Refs

• Anthropic Research mode engineering posts, OpenAI Deep Research blog, Stanford STORM paper, Plan-and-Execute patterns, LangGraph supervisor examples, Manus / GPT-Researcher repos.

FAQ

What is a multi-agent research system?

It is a design where a supervisor LLM decomposes a question into sub-tasks, dispatches parallel sub-agents to search and read sources, then synthesizes a cited report from their findings.

Why use multiple agents instead of one long-context LLM call?

Parallel sub-agents cut wall-clock time, isolate failure to one branch, and let each agent keep a focused context window, which beats a single bloated prompt on quality.

How do sub-agents share state safely?

They write findings to a shared scratchpad or memory store keyed by session, while the supervisor reads summaries and orchestrates next steps to avoid prompt collisions.

What latency and cost targets are realistic?

Typical research finishes in 1 to 5 minutes with 100 concurrent sessions, holding spend under about 1 dollar per session by capping sub-agent count and search calls.

How are citations gathered and verified?

Sub-agents store URLs and quoted snippets alongside answers, and the synthesizer attaches them to each claim in the final report so users can audit sources.

When should I pick a multi-agent design over a single agent?

Use it for open-ended research with many independent sub-questions; stick to a single agent for short, single-source queries where coordination overhead is not worth it.

Related Topics

• Agent Loop and ReAct: the reasoning loop each sub-agent runs
• Agent Memory: scratchpad, working memory, and citation graph
• RAG Patterns: retrieval over web and internal corpora during research