Skip to content

Container orchestration#

flowchart LR
  D[Developer]
  CI[CI builds image]
  REG[Image registry]
  K8S[Kubernetes control plane]
  N1[Node 1<br/>kubelet + pods]
  N2[Node 2<br/>kubelet + pods]
  N3[Node 3<br/>kubelet + pods]
  D --> CI --> REG --> K8S
  K8S --> N1
  K8S --> N2
  K8S --> N3

  classDef p fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
  classDef s fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  classDef e fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
  classDef d fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
  class D p;
  class CI s;
  class K8S e;
  class REG d;
  class N1,N2,N3 s;

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class D,CI,REG,K8S,N1,N2,N3 service;

Kubernetes orchestrates containers across a fleet: declare desired state, controllers reconcile to it, the scheduler places pods, kubelets run them, kube-proxy / Services / Ingress route traffic.

Architecture#

Kubernetes logo, the de facto container orchestration platform
Kubernetes logo, © The Linux Foundation / CNCF, via Wikimedia Commons.
flowchart TB
  subgraph CP[Control plane]
    API[API server]
    ETCD[(etcd<br/>cluster state)]
    SCHED[Scheduler]
    CTRL[Controllers<br/>deployment, replicaset,<br/>statefulset, job, hpa, ...]
  end

  subgraph N1[Node 1]
    KUBELET1[kubelet]
    PROXY1[kube-proxy]
    POD1[Pod A]
    POD2[Pod B]
  end

  subgraph N2[Node 2]
    KUBELET2[kubelet]
    PROXY2[kube-proxy]
    POD3[Pod C]
  end

  DEV[Developer / CI]
  DEV --> API
  API --- ETCD
  API --> SCHED
  API --> CTRL
  SCHED -.assign pods.-> KUBELET1
  SCHED -.assign pods.-> KUBELET2
  CTRL -.reconcile.-> KUBELET1
  CTRL -.reconcile.-> KUBELET2

  classDef cp fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
  classDef s fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  classDef d fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
  classDef p fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
  class API,SCHED,CTRL cp;
  class ETCD d;
  class KUBELET1,KUBELET2,PROXY1,PROXY2,POD1,POD2,POD3 s;
  class DEV p;

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class API,CTRL,KUBELET1,PROXY1,POD1,POD2,KUBELET2,PROXY2,POD3,DEV service;
    class ETCD datastore;
    class SCHED compute;

Core objects#

Object What it does
Pod smallest deployable unit: 1+ containers sharing network + storage
Deployment declarative rollouts of stateless workloads, replicas, update strategy
StatefulSet stable identity + ordered start/stop for stateful workloads
DaemonSet one pod per node (log agents, sidecars)
Job / CronJob run-to-completion workloads
Service stable virtual IP + DNS for a set of pods
Ingress L7 routing into the cluster from outside
ConfigMap / Secret injected config + secrets
PersistentVolumeClaim request for durable storage
HorizontalPodAutoscaler scale replicas on CPU / custom metrics
Namespace logical multi-tenancy
Operator / CRD extend k8s with custom resource + controller

How a deployment becomes traffic#

sequenceDiagram
  participant U as User
  participant Dev as kubectl apply
  participant API as kube-apiserver
  participant Sched as Scheduler
  participant Kube as kubelet
  participant Pod
  participant Svc as Service / kube-proxy
  Dev->>API: POST Deployment manifest
  API->>API: persist in etcd
  API->>Sched: new ReplicaSet
  Sched->>Kube: bind pod to node
  Kube->>Pod: pull image, run container
  Pod-->>Svc: ready (passes readiness probe)
  U->>Svc: HTTP request
  Svc->>Pod: forward (iptables / IPVS / eBPF)

Reconciliation loop#

  • You declare desired state in a YAML manifest.
  • The controller reads desired vs actual.
  • Controller takes action (create pod, kill pod, scale).
  • Repeats forever - self-healing on failure.

Networking model#

  • Every pod gets its own IP - flat network, no NAT between pods.
  • Services = stable virtual IP + DNS name → pods (selected by label).
  • kube-proxy programs iptables / IPVS / eBPF (Cilium) for service IPs.
  • Ingress = L7 routing (Nginx / Envoy / ALB-controller).
  • CNI plugins (Calico, Cilium, Flannel) implement the network.

Storage#

  • Ephemeral by default. Mount a PVC to get durable storage.
  • StorageClass maps to a backend (EBS, GCE PD, NFS, Rook/Ceph).
  • StatefulSets give each pod a stable PVC.

Observability#

  • kubectl logs / describe / get events.
  • Metrics via Prometheus + kube-state-metrics + node-exporter.
  • Traces via OpenTelemetry collectors.
  • See observability.

Common interview hooks#

  • "How does a pod talk to a database in another namespace?" → Service DNS (db.payments.svc.cluster.local) + NetworkPolicy.
  • "How do you do zero-downtime deploy?" → rolling update, readiness probes, surge / unavailable settings.
  • "Why StatefulSet over Deployment?" → stable identity + ordered start; pods are not interchangeable.
  • "What if etcd loses quorum?" → cluster goes read-only; running pods keep working but no new scheduling.
  • "How do you handle secrets?" → External Secrets Operator → Vault / KMS; never commit Secret YAML.

When K8s is overkill#

  • Single service, single deploy target → use ECS / Fargate / Cloud Run / Render.
  • < 5 services → docker-compose on one box might be enough.

Glossary & fundamentals#

Tag Concept Page
HLD Service discovery (k8s does it for you) service-discovery
HLD Service mesh (east-west between pods) service-mesh
HLD Load balancer (Service + Ingress) load-balancer
HLD Observability (k8s-native) observability
HLD Multi-region & DR (cluster federation, fleet) multi-region-dr

Quick reference#

Day-1 mental model#

  • Container = single packaged process (Docker / OCI image).
  • Pod = group of containers sharing network + storage.
  • Node = a VM running kubelet, hosts pods.
  • Cluster = set of nodes managed by a control plane.

Probes#

  • Liveness - "is the process alive?" → restart if not.
  • Readiness - "can I serve traffic?" → remove from Service if not.
  • Startup - "give the app a long initial grace period" before liveness kicks in.

Capacity & scaling#

  • Resource requests drive scheduling; limits drive enforcement.
  • HPA for replica scaling on CPU / RPS.
  • VPA for right-sizing a single pod.
  • Cluster autoscaler for adding / removing nodes.

Day-2 ops#

  • GitOps (Argo CD / Flux) for declarative deploys.
  • Helm / Kustomize for templating.
  • Policy enforcement with OPA Gatekeeper or Kyverno.
  • Secrets via External Secrets / Sealed Secrets.

Refs#

  • Kubernetes docs → see Further reading.
  • "Kubernetes Up & Running" - Burns, Beda, Hightower.
  • KubeCon talks on production patterns.
  • Brendan Burns: "The History of Kubernetes".

FAQ#

What is container orchestration?#

Container orchestration automates deployment, scaling, networking, and lifecycle of containers across a cluster of machines. The orchestrator schedules workloads, restarts failed containers, and rolls out updates.

What is the difference between Kubernetes and Docker Swarm?#

Kubernetes is the de facto standard with a richer feature set, larger ecosystem, and broad cloud support. Swarm is simpler to operate and tightly coupled to Docker, but it has been declining in adoption.

What does the Kubernetes control plane do?#

It runs the API server, scheduler, controllers, and etcd. The scheduler assigns pods to nodes, controllers reconcile desired and actual state, and etcd stores cluster state durably.

When should I pick Nomad over Kubernetes?#

Pick Nomad for smaller teams who want a single binary, easy operations, and support for non-container workloads like raw binaries or VMs. Pick Kubernetes for ecosystem depth and cloud-native tooling.

How does Kubernetes schedule pods?#

The scheduler filters nodes by resource requests, affinities, taints, and topology constraints, then scores remaining candidates. The highest-scoring node is selected and the kubelet starts the pod.

  • Service Mesh: service meshes run on top of container orchestration platforms to manage service communication
  • Observability: container orchestration generates metrics, logs, and traces that observability systems must handle
  • API Gateway: API gateways are commonly deployed as orchestrated workloads and integrate with ingress controllers

Further reading#

Curated, high-credibility sources for going deeper on this topic.