Skip to content

Code Execution Platform (LeetCode / Replit)#

Problem statement (interviewer prompt)

Design a code execution platform (LeetCode / Replit / Judge0): users submit code in any of 30+ languages, you compile + run it in a sandbox against test cases, return the result in seconds, and prevent abuse (CPU/memory/network/file-system limits).

flowchart LR
  U([User])
  API[Submit API]
  Q[[Job Queue]]
  WORK([Sandbox Worker])
  RES[Result store]
  U --> API --> Q --> WORK --> RES --> U

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class U client;
    class API service;
    class RES datastore;
    class Q queue;
    class WORK compute;

flowchart TB
  subgraph Client
    UI([Web IDE])
  end

  subgraph API
    SUB[Submit endpoint]
    AUTH[Auth + rate limit]
    VAL[Lang + version validate]
  end

  subgraph Queue
    Q[[Priority queue]]
    DEAD[(DLQ)]
  end

  subgraph Sandbox[Sandbox executor]
    POOL([Worker pool])
    DOCKER[Container per submission]
    FIRECRACKER[Firecracker microVM<br/>strong isolation]
    SECCOMP[seccomp + namespaces]
    CGROUPS[CPU / memory / time limits]
    NETN[Network isolation]
    FS[Read-only FS + tmpfs scratch]
    LANG[Language runtimes + deps]
  end

  subgraph TestRun[Test runner]
    INP[Stdin inputs]
    EXEC[Run with timeout]
    OUT[Capture stdout/stderr]
    DIFF[Compare expected]
    SCORE[Score / partial credit]
  end

  subgraph Storage
    SUBS[(Submissions)]
    PROB[(Problems / hidden tests)]
    LDB[(Leaderboards)]
    SOLS[(Per-user solutions)]
  end

  subgraph Ops
    SCALE([Autoscale workers])
    METR[[Time-per-run, queue lag]]
    ABUSE[Abuse / fork bomb detection]
  end

  Client --> API --> Queue --> Sandbox --> TestRun --> Storage
  Ops --- Sandbox

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class UI client;
    class SUB,AUTH,VAL,DOCKER,FIRECRACKER,SECCOMP,CGROUPS,NETN,FS,LANG,INP,EXEC,OUT,DIFF,SCORE,ABUSE service;
    class DEAD,SUBS,PROB,LDB,SOLS datastore;
    class Q,METR queue;
    class POOL,SCALE compute;

Isolation choices#

  • Containers (Docker) = fast, weaker isolation.
  • MicroVMs (Firecracker / gVisor) = stronger; sub-100 ms start.
  • One-shot ephemeral per submission; no cross-submission state.

Resource limits#

  • CPU cgroup, memory cgroup, wall-clock + cpu-time limits.
  • seccomp filters disallowed syscalls.
  • No network unless explicitly needed.

Glossary & fundamentals#

Concepts referenced in this design. Each row links to its canonical page; the tag column shows whether it is a high-level (HLD) or low-level (LLD) concept.

Tag Concept What it is Page
HLD Pub/Sub & message brokers topics, consumer groups, delivery semantics pub-sub-pattern

Quick reference#

Functional#

  • Compile + run user code in many languages.
  • Run against test cases; score.
  • Stream output back.

Non-functional#

  • p99 < 3 s typical contest run.
  • Isolation against fork bombs, network probes, side channels.
  • 99.95% uptime.

Capacity#

  • Contest spikes: tens of thousands of submissions/min.
  • Worker fleet autoscaled to queue depth.

Trade-offs#

  • Containers = fast cold start; microVMs = stronger isolation.
  • Warm pool of pre-launched runtimes saves startup latency.
  • Hidden tests must remain server-side; never ship to client.

Refs#

  • Firecracker (AWS) paper.
  • HackerRank / LeetCode / Codeforces engineering posts.
  • gVisor / Kata Containers docs.
  • Replit / CodeSandbox architecture posts.

FAQ#

How does an online judge run user code safely?#

Submitted code runs in a short-lived sandbox (Firecracker microVM or gVisor container) with CPU, memory, network, and filesystem limits enforced by the kernel. The host never executes user code directly.

MicroVM vs container for code execution?#

MicroVMs (Firecracker) give hardware-level isolation suitable for multi-tenant untrusted code. Containers are faster to start but share the host kernel, so they need seccomp and namespaces to be safe.

How do you stream output back to the user?#

Attach to the sandbox's stdout/stderr file descriptors, forward bytes over WebSocket or SSE, and flush at a small interval. The user sees output as the program prints it.

How do you enforce CPU and memory limits?#

Use cgroups to cap CPU shares and memory, and kill the sandbox if the wall clock exceeds the budget. On OOM the kernel signals the container and the worker reports a memory-limit error.

How do you scale to many concurrent submissions?#

Submissions go into a job queue. A pool of stateless workers pulls jobs, runs each in a fresh sandbox, and writes the verdict to a result store. Autoscale workers on queue depth.

Video walkthrough

Design LeetCode (Online Judge) : via Hello Interview