Containers vs VMs

The containers vs virtual machines question is really about where the isolation boundary lives. A VM runs its own full OS kernel on top of a hypervisor (KVM, Xen, ESXi). A container shares the host kernel and gets fenced off by Linux namespaces and cgroups. That single difference cascades into everything: startup time, density, security blast radius, and the right runtime to pick.

Docker logo, © Docker, Inc., via Wikimedia Commons.

Problem statement (interviewer prompt)

Compare containers and virtual machines as compute units. Cover shared kernel vs hypervisor isolation, namespaces and cgroups, startup time and density, the security trade-off, and modern runtimes (runc, gVisor, Kata, Firecracker). When does each win in Kubernetes pods?

flowchart LR
  subgraph Host[Bare Metal Host]
    HW[Hardware]
    KER[Host Kernel]
  end

  subgraph VMs[Virtual Machines]
    HYP[Hypervisor]
    K1[Guest Kernel A]
    K2[Guest Kernel B]
    APP1[App in VM A]
    APP2[App in VM B]
  end

  subgraph CTR[Containers]
    NS[Namespaces + cgroups]
    C1[App C1]
    C2[App C2]
    C3[App C3]
  end

  HW --> KER
  KER --> HYP --> K1 --> APP1
  HYP --> K2 --> APP2
  KER --> NS --> C1
  NS --> C2
  NS --> C3

  classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
  classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
  classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  class HW,KER storage;
  class HYP,K1,K2,NS service;
  class APP1,APP2,C1,C2,C3 compute;

VMs trade density and startup time for a strong, hardware-enforced isolation boundary, which is why public clouds use them as the tenant boundary. Containers trade isolation strength for sub-second start and 10x density, which is why microservice fleets and CI runners ride on Kubernetes pods. Modern runtimes (gVisor, Kata, Firecracker) blur the line: container ergonomics with VM-grade isolation.

Problem statement (interviewer prompt)

You are sizing compute for a multi-tenant SaaS. Compare containers and virtual machines as the tenant boundary. Cover hypervisor types, Linux namespaces and cgroups, density and startup time, the security blast radius, and the role of runc, gVisor, Kata Containers, and Firecracker. Where does each model fit in a Kubernetes pod?

The isolation primitives

A virtual machine boots its own kernel on top of a hypervisor. Type-1 hypervisors (KVM, Xen, ESXi, Hyper-V) run directly on the hardware and use CPU features (Intel VT-x, AMD-V) to trap privileged instructions. Type-2 (VirtualBox, Parallels) run inside a host OS for desktop use. Each VM has its own kernel, drivers, and full memory image; the boundary is enforced by the CPU plus the hypervisor.

A container is just a Linux process with a custom view of the world, assembled from:

Primitive	Isolates
mnt namespace	Filesystem view
pid namespace	Process IDs
net namespace	Network interfaces, ports, routes
uts namespace	Hostname, domain
ipc namespace	Shared memory, semaphores
user namespace	UID/GID mappings, capabilities
cgroup namespace	cgroup hierarchy view
time namespace	Clock
cgroups v2	CPU, memory, IO, PID quotas
seccomp-bpf	Filtered syscalls
LSM (AppArmor, SELinux)	Mandatory access control
capabilities	Drop privileged bits

The host kernel sits underneath all of these. A kernel bug or a misconfigured seccomp profile is enough to break out.

flowchart TB
  subgraph VM
    HV[Hypervisor: KVM/Xen]
    GK[Guest Kernel]
    APP[App + libs]
  end

  subgraph Container
    HK[Host Kernel]
    NS[Namespaces]
    CG[cgroups]
    SEC[seccomp + LSM]
    CAPP[App + libs]
  end

  HV --> GK --> APP
  HK --> NS --> CAPP
  HK --> CG --> CAPP
  HK --> SEC --> CAPP

  classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
  classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
  classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  class HV,GK,HK storage;
  class NS,CG,SEC service;
  class APP,CAPP compute;

Quantitative trade-offs

Dimension	Container	VM
Boot time	50 ms - 2 s	10 s - 60 s (1-2 s with Firecracker)
Image size	5 MB - 1 GB	1 GB - 10 GB
Density per host	100-1000	10-50
Memory overhead	KB per container	100-500 MB per VM (kernel + page tables)
Isolation boundary	Kernel + namespaces	CPU + hypervisor
Live migration	Limited (CRIU)	Mature
OS choices	Same host kernel	Any guest OS
Pet vs cattle	Cattle	Either

Runtime spectrum

Modern Kubernetes lets you pick a runtime per pod via RuntimeClass. The OCI-compliant runtime contracts make these largely drop-in.

flowchart LR
  RUNC[runc<br/>plain containers]
  GV[gVisor<br/>userspace kernel]
  KATA[Kata Containers<br/>QEMU micro-VM]
  FC[Firecracker<br/>minimal VMM]
  STD[Standard VM<br/>KVM + qemu]

  RUNC -- weakest isolation --> ISO[Isolation strength]
  GV --> ISO
  KATA --> ISO
  FC --> ISO
  STD -- strongest isolation --> ISO

  classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
  class RUNC,GV,KATA,FC,STD service;
  class ISO compute;

• runc: reference OCI runtime. Pure namespaces + cgroups. Fastest, weakest isolation.
• gVisor: intercepts syscalls in a Go-based userspace kernel ("Sentry"), reducing attack surface but with 10-30% performance penalty on syscall-heavy workloads.
• Kata Containers: lightweight QEMU micro-VM per pod, sharing little with the host. Container ergonomics, VM boundary.
• Firecracker: stripped-down VMM (no PCI, no BIOS, no GPU). Powers AWS Lambda and Fargate; boots in ~125 ms. Used via firecracker-containerd.
• Standard VMs (KVM, ESXi): when the tenant boundary is a customer or compliance requirement (PCI, HIPAA).

Security blast radius

A container escape needs a kernel exploit (CVE-2022-0185, runc CVE-2019-5736 "leaky" symlink, recent Dirty Pipe variants). Once out, the attacker owns the host and every co-tenant container. A VM escape needs a hypervisor exploit, which is rarer and patched aggressively because the cloud business depends on it.

That asymmetry drives the rule of thumb: trust boundary = at least a VM, plus containers for density inside it. Public clouds run one VM per customer per host and pack their containers inside; you should too if running untrusted code (CI runners, customer functions, sandboxes).

Kubernetes pods

A pod is a group of containers sharing net, ipc, uts, and optionally pid namespaces, plus mounted volumes. Each container still has its own mnt namespace and cgroup. A pod's lifecycle is tied to a node; for stronger isolation, schedule the pod onto a node running gVisor or Kata via RuntimeClass:

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc
---
apiVersion: v1
kind: Pod
metadata:
  name: untrusted-job
spec:
  runtimeClassName: gvisor
  containers:
    - name: app
      image: ghcr.io/example/job:1.2

When each model wins

flowchart TB
  WL[Workload]
  WL -->|trusted, dense, fast-cycle| CTR[Containers / runc]
  WL -->|untrusted code, sandbox| SVC[gVisor or Kata]
  WL -->|multi-tenant SaaS perimeter| VM[Standard VM]
  WL -->|FaaS sub-second cold start| FC[Firecracker micro-VM]
  WL -->|legacy OS, kernel modules| FULLVM[Full VM]

  classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
  class WL service;
  class CTR,SVC,VM,FC,FULLVM compute;

• Containers win when you run your own trusted code, deploy often, and want to pack many services per node.
• Strong-isolation containers (gVisor, Kata) win for untrusted user-uploaded code, multi-tenant build runners, and security sandboxes.
• VMs win as the customer / blast-radius boundary in multi-tenant SaaS and clouds, for OS variety, and for legacy workloads needing kernel modules.
• Firecracker wins where you need VM-grade isolation but FaaS-grade density and start time (AWS Lambda, Fargate, Fly.io).

Where it is used

• Kubernetes (runc by default, optional Kata or gVisor RuntimeClass).
• AWS Fargate, Lambda: Firecracker micro-VMs.
• Google Cloud Run: gVisor for second-gen, lightweight VMs for first-gen execution environment.
• Docker Desktop: a Linux VM hosting containers on macOS/Windows.
• GitHub Actions runners: VM per job for tenant isolation.

Quick reference

Core difference

• VM: own guest kernel on a hypervisor (KVM, Xen, ESXi). Hardware-enforced boundary.
• Container: process with custom namespaces + cgroups on the host kernel. Software-enforced boundary.

Container primitives

• Namespaces: mnt, pid, net, uts, ipc, user, cgroup, time.
• cgroups v2: CPU, memory, IO, PID quotas.
• seccomp-bpf: syscall allowlist.
• LSM (AppArmor, SELinux): mandatory access control.
• Linux capabilities: drop root bits like CAP_NET_ADMIN.

Quick comparison

	Container	VM
Boot	50 ms - 2 s	10-60 s; Firecracker ~125 ms
Image	5 MB - 1 GB	1-10 GB
Density / host	100-1000	10-50
Memory overhead	KB	100-500 MB
Isolation	namespaces + cgroups	CPU + hypervisor
OS	shared host kernel	any guest OS

Runtime spectrum

• runc: reference OCI runtime, weakest isolation, fastest.
• gVisor (runsc): userspace kernel intercepts syscalls; 10-30% perf hit on syscall-heavy work.
• Kata Containers: QEMU micro-VM per pod, VM boundary with container ergonomics.
• Firecracker: stripped VMM, ~125 ms boot. Powers AWS Lambda, Fargate, Fly.io.
• Full VM (KVM, ESXi): customer / compliance boundary.

Security

• Container escapes need a kernel CVE; one escape = whole host owned.
• VM escapes need a hypervisor CVE; rarer, harder.
• Rule: trust boundary at VM, density via containers inside.
• Run untrusted code via gVisor or Kata.

Kubernetes

• Pod = group of containers sharing net, ipc, uts, optional pid.
• RuntimeClass selects runc, gVisor, or Kata per pod.
• securityContext: drop capabilities, set readonly rootfs, non-root user.

When each wins

• Containers: trusted services, fast deploy, high density.
• gVisor/Kata: untrusted code, build runners, sandboxes.
• Firecracker: FaaS-grade density + VM isolation.
• Full VMs: tenant perimeter, legacy OS, kernel modules, GPU passthrough.

Refs

• Agache et al.: "Firecracker - Lightweight Virtualization for Serverless Applications" (NSDI 2020).
• gVisor architecture guide.
• Kata Containers design docs.
• man 7 namespaces, man 7 cgroups.

FAQ

What is the difference between a container and a VM?

A VM runs its own full OS on a hypervisor. A container shares the host kernel and is isolated by namespaces and cgroups. Containers start in milliseconds and pack denser; VMs offer stronger isolation.

Are containers less secure than VMs?

Yes by default, because all containers on a host share the kernel, so a kernel exploit escapes to all of them. VMs cross the hypervisor boundary, which is smaller and better hardened in practice.

What is Kata Containers?

Kata Containers run each container inside a lightweight VM using Firecracker or QEMU, giving VM-grade isolation with container-grade UX. They are useful for multi-tenant workloads on shared nodes.

What is gVisor?

gVisor is a user-space kernel that intercepts container syscalls and re-implements them in Go, reducing the host kernel attack surface. It is slower than runc but safer for untrusted code.

When should I use VMs over containers?

Use VMs when running untrusted workloads, multi-tenant SaaS without sandboxing, legacy software needing a different kernel, or when regulations require hypervisor-level isolation.

Related Topics

• Container Orchestration: Kubernetes schedules pods (groups of containers) and lets you pick runtimes per workload.
• Serverless / FaaS: FaaS platforms run code inside Firecracker micro-VMs to combine VM isolation with container density.
• Multi-Tenancy Patterns: the container vs VM choice maps directly to soft vs hard tenant isolation.

Further reading

• Paper - Agache et al. - Firecracker: Lightweight Virtualization for Serverless Applications (NSDI '20)
• Docs - gVisor architecture overview
• Docs - Kata Containers architecture
• Docs - Linux namespaces (man 7 namespaces)
• Blog - Julia Evans - What even is a container?