What’s the actual difference between a container and a microVM for running untrusted code?

A container is a normal process with Linux namespaces narrowing what it can see and cgroups capping what it can use — but every container on the box still asks the exact same host kernel to do anything privileged, so a bug in that one shared kernel is a bug against every tenant at once. A microVM like Firecracker gives each sandbox its own guest kernel, running on top of a hardware virtualization boundary (KVM) rather than a software one — Firecracker’s own design doc puts it plainly: “the first layer of isolation is provided by the Linux KVM and the Firecracker virtualization boundary.” A kernel exploit inside one microVM only ever reaches that microVM’s own disposable guest kernel, never the host’s. The price is a boot: Firecracker “initiates user space or application code in as little as 125 ms,” versus a container’s near-instant fork — small in absolute terms, but the reason production systems bother with warm pools and snapshots at all.

How does gVisor sandbox code without running a full virtual machine?

gVisor puts a second kernel — written in Go, running entirely in user space — between the sandboxed application and the real Linux kernel. Every syscall the sandboxed code makes (the narrow interface a process uses to ask the kernel to open a file, spawn a thread, or touch the network) is intercepted and handled by that user-space kernel, called the Sentry, instead of being passed straight through. gVisor’s own docs describe it as intercepting “application system calls and act[ing] as the guest kernel, without the need for translation through virtualized hardware.” Modal, which builds its Sandboxes on gVisor, reports that the Sentry itself “restricts its own host interactions to roughly 68 allowlisted syscalls out of over 350 in the Linux kernel” — so even a compromised Sentry can only ask the real kernel to do a tiny, audited set of things. The tradeoff is per-syscall overhead, which gVisor’s docs admit outright: it “comes at the price of reduced application compatibility and higher per-system call overhead.”

Why do serverless platforms use VM snapshots instead of just booting fast?

Because the expensive part isn’t the virtual machine’s own boot — it’s everything that happens inside the guest afterward: booting a kernel, starting an init process, loading a language runtime, importing packages, warming an interpreter. AWS Lambda’s SnapStart captures a microVM’s full memory and disk state after that entire initialization has already finished, then restores new invocations directly from that captured state instead of repeating the work. AWS states the effect directly: it helps “reduce variable cold start latency from several seconds (or higher) to as low as sub-second.” Under the hood the snapshot is split into 512 KB chunks served from a cache tier AWS says is “typically 1 millisecond for a 512 KB chunk” from the fastest layer — a restore is a bulk memory copy, not a rerun of the entire boot-and-initialize sequence.

How do you stop a fork bomb or an infinite loop inside a sandbox?

Two different mechanisms, because they’re two different failure modes. A fork bomb — code that endlessly spawns copies of itself — is stopped by the Linux kernel’s pids cgroup controller, whose own documentation states its purpose plainly: it lets a cgroup “stop any new tasks from being fork()’d or clone()’d after a specified limit is reached,” precisely because “a fork bomb is likely to exhaust the number of tasks before hitting memory restrictions.” An infinite loop that never forks and never allocates much memory is a different problem — it can sit comfortably under every cgroup limit forever. That’s caught by a wall-clock supervisor outside the sandbox entirely: a timer that force-kills the whole sandbox at a hard deadline no matter what it’s doing internally. Judge0, an open-source code-execution sandbox, ships exactly this as a first-class feature: “custom user-defined compiler options, command-line arguments, and time and memory limits.”

Should a code-execution sandbox be allowed to access the internet?

Not by default, and if it must, only through a proxy with an explicit allowlist. An unrestrained outbound connection turns a perfectly isolated compute sandbox into an exfiltration channel for whatever secrets it was handed, a scanner for whatever internal network the host sits on, or free infrastructure for a botnet — none of which requires ever touching the kernel boundary at all. Real platforms vary: Modal’s docs are candid that “by default, Sandboxes can make outbound connections to any public IP address,” but layer on an outbound domain allowlist for tenants that need it tighter, and separately guarantee that regardless of network policy, “Sandboxes are not authorized to access other resources in your Modal workspace... the blast radius of any malicious code will be limited to the Sandbox container itself.” The safer default for a genuinely untrusted, agent-driven workload is to start with no egress at all and open specific, named destinations through a proxy — not the other way around.

Design a Sandboxed Code-Execution Service

Run code an LLM wrote seconds ago, for thousands of mutually suspicious tenants, without ever letting one read another’s files or take the host down — the isolation spectrum from a shared process to a hardware-walled microVM, warm pools and snapshot/restore, cgroup limits, and network policy, priced in real milliseconds and megabytes from the Firecracker and gVisor docs.

System design · Systems. The source ↗

A free, interactive, animated visual explainer of Design a Sandboxed Code-Execution Service — built to be understood, not skimmed.

Questions

What’s the actual difference between a container and a microVM for running untrusted code?: A container is a normal process with Linux namespaces narrowing what it can see and cgroups capping what it can use — but every container on the box still asks the exact same host kernel to do anything privileged, so a bug in that one shared kernel is a bug against every tenant at once. A microVM like Firecracker gives each sandbox its own guest kernel, running on top of a hardware virtualization boundary (KVM) rather than a software one — Firecracker’s own design doc puts it plainly: “the first layer of isolation is provided by the Linux KVM and the Firecracker virtualization boundary.” A kernel exploit inside one microVM only ever reaches that microVM’s own disposable guest kernel, never the host’s. The price is a boot: Firecracker “initiates user space or application code in as little as 125 ms,” versus a container’s near-instant fork — small in absolute terms, but the reason production systems bother with warm pools and snapshots at all.
How does gVisor sandbox code without running a full virtual machine?: gVisor puts a second kernel — written in Go, running entirely in user space — between the sandboxed application and the real Linux kernel. Every syscall the sandboxed code makes (the narrow interface a process uses to ask the kernel to open a file, spawn a thread, or touch the network) is intercepted and handled by that user-space kernel, called the Sentry, instead of being passed straight through. gVisor’s own docs describe it as intercepting “application system calls and act[ing] as the guest kernel, without the need for translation through virtualized hardware.” Modal, which builds its Sandboxes on gVisor, reports that the Sentry itself “restricts its own host interactions to roughly 68 allowlisted syscalls out of over 350 in the Linux kernel” — so even a compromised Sentry can only ask the real kernel to do a tiny, audited set of things. The tradeoff is per-syscall overhead, which gVisor’s docs admit outright: it “comes at the price of reduced application compatibility and higher per-system call overhead.”
Why do serverless platforms use VM snapshots instead of just booting fast?: Because the expensive part isn’t the virtual machine’s own boot — it’s everything that happens inside the guest afterward: booting a kernel, starting an init process, loading a language runtime, importing packages, warming an interpreter. AWS Lambda’s SnapStart captures a microVM’s full memory and disk state after that entire initialization has already finished, then restores new invocations directly from that captured state instead of repeating the work. AWS states the effect directly: it helps “reduce variable cold start latency from several seconds (or higher) to as low as sub-second.” Under the hood the snapshot is split into 512 KB chunks served from a cache tier AWS says is “typically 1 millisecond for a 512 KB chunk” from the fastest layer — a restore is a bulk memory copy, not a rerun of the entire boot-and-initialize sequence.
How do you stop a fork bomb or an infinite loop inside a sandbox?: Two different mechanisms, because they’re two different failure modes. A fork bomb — code that endlessly spawns copies of itself — is stopped by the Linux kernel’s pids cgroup controller, whose own documentation states its purpose plainly: it lets a cgroup “stop any new tasks from being fork()’d or clone()’d after a specified limit is reached,” precisely because “a fork bomb is likely to exhaust the number of tasks before hitting memory restrictions.” An infinite loop that never forks and never allocates much memory is a different problem — it can sit comfortably under every cgroup limit forever. That’s caught by a wall-clock supervisor outside the sandbox entirely: a timer that force-kills the whole sandbox at a hard deadline no matter what it’s doing internally. Judge0, an open-source code-execution sandbox, ships exactly this as a first-class feature: “custom user-defined compiler options, command-line arguments, and time and memory limits.”
Should a code-execution sandbox be allowed to access the internet?: Not by default, and if it must, only through a proxy with an explicit allowlist. An unrestrained outbound connection turns a perfectly isolated compute sandbox into an exfiltration channel for whatever secrets it was handed, a scanner for whatever internal network the host sits on, or free infrastructure for a botnet — none of which requires ever touching the kernel boundary at all. Real platforms vary: Modal’s docs are candid that “by default, Sandboxes can make outbound connections to any public IP address,” but layer on an outbound domain allowlist for tenants that need it tighter, and separately guarantee that regardless of network policy, “Sandboxes are not authorized to access other resources in your Modal workspace... the blast radius of any malicious code will be limited to the Sandbox container itself.” The safer default for a genuinely untrusted, agent-driven workload is to start with no egress at all and open specific, named destinations through a proxy — not the other way around.

Design a Sandboxed Code-Execution Service

Questions

Related explainers