Code Execution › Sandboxed Execution

Running LLM-generated code is inherently risky. The model has no guarantee of correctness, and its outputs may accidentally or intentionally perform dangerous operations: deleting files, making network requests, consuming excessive memory, or running infinite loops. Sandboxed execution addresses these risks by running code in an isolated environment with limited privileges and resources.

The need for sandboxing predates LLMs significantly. Online judges for competitive programming, browser JavaScript engines, server-side scripting environments, and cloud function platforms all face the same challenge: running arbitrary code submitted by untrusted parties without endangering the host system. The solutions developed for these use cases form the technical foundation for LLM code execution environments.

A sandbox is a controlled execution environment that restricts what a program can do. The fundamental idea is to run untrusted code inside a container of constraints so that even if the code behaves badly, the damage is contained. Sandboxing operates at multiple levels, from lightweight process isolation all the way to full virtual machine separation.

Think of a sandbox as a theater stage with hard walls. The actors on stage can do anything within the stage area, but the stage manager at the controls decides what props are available, whether they can walk into the audience, and what happens when the scene ends. The audience (the host system) watches from a safe distance, and nothing that happens on stage can reach them unless the stage manager explicitly allows it.

In the context of LLM code execution, sandboxing typically involves several mechanisms working together:

• Process isolation: Running generated code in a separate process with a restricted user account, so it cannot access files or processes belonging to the calling application.
• Resource limits: Imposing limits on CPU time, memory, and disk I/O using OS-level mechanisms like ulimit on Linux or cgroups and namespaces.
• Filesystem restrictions: Mounting a read-only filesystem or a temporary writable directory that gets wiped after execution.
• Network isolation: Blocking outbound network connections so code cannot exfiltrate data, download malware, or make unauthorized API calls.
• System call filtering: Using mechanisms like seccomp (secure computing mode) to whitelist only the system calls the sandbox needs, blocking everything else.

Each of these mechanisms targets a different attack surface. Process isolation prevents inter-process interference. Resource limits prevent denial-of-service through resource exhaustion. Filesystem restrictions prevent data exfiltration or persistent modification. Network isolation prevents external communication. System call filtering is the deepest defense: it prevents the process from even asking the kernel to do things outside its allowed scope.

The most widely deployed approach for sandboxing code execution at scale is containerization using tools like Docker or Podman. A container packages a minimal runtime environment (Python interpreter, standard libraries, and nothing else) into an isolated unit. The container shares the host kernel but has its own filesystem, network namespace, and process namespace.

Docker's layered filesystem is particularly useful for code execution. The base image containing the Python runtime is shared immutably across all executions. Each execution gets a thin writable layer on top, which is discarded when the container exits. This means spin-up cost is minimal (no copying files) and cleanup is trivially fast (just discard the writable layer).

When an LLM generates code that needs to execute, the execution engine follows a consistent sequence:

1. Write the generated code to a temporary file inside a fresh container instance.
2. Invoke the interpreter inside that container with resource limits applied.
3. Capture stdout, stderr, and the exit code.
4. Destroy the container after execution completes or a timeout fires.

Container startup latency is a concern for interactive use cases. A cold Docker container can take hundreds of milliseconds to start. Systems that need fast feedback (like REPL-style notebooks or real-time coding assistants) often keep a pool of pre-warmed container instances ready to accept code, trading memory for lower latency. The pool approach is analogous to a web server keeping a pool of worker processes: rather than spawning a new process for each request (slow), the server maintains a set of ready workers and dispatches requests to them.

Docker containers share the host kernel, which means a kernel exploit in the contained code could escape the sandbox. The container isolation boundary is the namespace layer; if an attacker finds a kernel vulnerability (and kernel CVEs do appear regularly), the container is no protection. For higher-assurance environments, tools like gVisor (from Google) or Firecracker (from AWS) provide stronger isolation.

gVisor implements a user-space kernel in Go that intercepts all system calls from the container, re-implementing them in a sandboxed guest kernel rather than passing them through to the host. The cost is higher overhead per system call, because every syscall must traverse the user-space kernel rather than going directly to the host. But the gain is that even a successful kernel exploit only compromises the guest kernel, not the host. From the perspective of the untrusted code, it sees a normal Linux kernel API; it just does not know that the "kernel" is a Go program running in user space.

Firecracker takes a different approach, spinning up a minimal microVM for each execution. The isolation is at the hypervisor level, which is the strongest available short of physical machine separation. AWS Lambda uses Firecracker under the hood, which is one reason it can safely run arbitrary customer code with strong isolation guarantees. The tradeoff is that microVM cold starts take longer (typically 100-150 milliseconds) and memory overhead is higher, since each execution needs its own kernel and minimal OS.

The choice between gVisor and Firecracker often comes down to the required combination of startup speed, memory budget, and security posture. Firecracker offers stronger isolation but higher startup cost; gVisor offers faster startup with somewhat weaker (though still strong) isolation.

WebAssembly (Wasm) has emerged as another sandboxing mechanism, particularly for lightweight, polyglot execution. Code compiled to Wasm runs in a deterministic sandbox with explicit memory bounds and no direct access to OS interfaces. The WebAssembly System Interface (WASI) provides a capability-based API that gives Wasm modules access only to the resources they are explicitly granted.

The security model of WebAssembly is unusually principled. Rather than trying to restrict a general-purpose process through monitoring and interception, WebAssembly simply does not have the primitives needed for dangerous operations. A Wasm module cannot make arbitrary syscalls; it can only call the specific functions exposed by the host. Trying to access a file that was not explicitly handed to the module is not a policy violation that gets caught at runtime; it is literally not expressible in the Wasm instruction set.

For LLM code execution, Wasm sandboxes are attractive because they provide strong isolation, fast startup times (microseconds, not milliseconds), and support for multiple languages. The tradeoff is that not all Python packages compile to Wasm, and the execution environment is more restricted than a full Linux container. Projects like Pyodide (CPython compiled to Wasm) have expanded Python support significantly, but numerical packages with C extensions remain challenging.

Even in a well-isolated sandbox, runaway code can exhaust resources and cause denial-of-service conditions for other users sharing the infrastructure. Every production execution environment enforces hard limits:

• Wall-clock timeout: The most important limit. Any execution that has not completed within a threshold (commonly 5-30 seconds for interactive use, up to minutes for batch jobs) is forcibly terminated. This handles infinite loops and long-running computations.
• Memory limit: Prevents unbounded memory allocation. When a process exceeds its memory cap, the OS sends SIGKILL. This catches accidental or intentional memory exhaustion.
• CPU quota: Limits the fraction of CPU time the sandbox can consume, preventing one execution from starving others on shared infrastructure.
• File size limits: Caps the size of files that generated code can write, preventing disk exhaustion.
• Process count limits: Prevents fork bombs, where code spawns an exponentially growing tree of child processes.

Setting these limits is both an art and a science. Too tight, and legitimate programs timeout or run out of memory. Too loose, and a single bad execution can degrade service for everyone else. The right values depend on the expected workload: a coding assistant helping with algorithmic problems needs different limits than one helping with data science tasks that process large files.

A special case worth discussing separately is the Jupyter notebook execution model, because it is the environment where many data scientists and researchers interact with LLM-generated code. Jupyter executes code in a persistent kernel: a Python interpreter that maintains state between cells. Variables defined in one cell are available in the next. Libraries imported in one cell stay imported throughout the session.

This stateful, incremental execution model is both powerful and tricky for LLM code generation. On the positive side, a model generating code for a notebook can assume that prior cells have already run, meaning it does not need to re-import libraries or redefine data that the notebook has already established. On the negative side, the notebook's state at any given moment depends on the execution history, which may not be apparent from the cell contents alone (if cells were run out of order, re-run multiple times, or produce non-deterministic outputs).

Systems like GitHub Copilot's notebook integration and tools like Cursor handle this by extracting the visible code context above the current cell and including it in the generation prompt. This gives the model the same approximate view of the notebook state that the user has. But it is an approximation: cells that were run but then deleted, cells whose outputs were cleared, and the current values of mutable objects are all invisible to the model from code text alone.

The statefulness also affects error handling. When a cell fails in a Jupyter notebook, the variables that were supposed to be created by that cell are missing, which cascades into failures in subsequent cells. A self-correcting system in the Jupyter context needs to understand this dependency structure, knowing that fixing a failing cell may require re-running earlier cells to restore state.