TechPrufer
Get started
Modal
Modal logoModal · Feature trail

Sandboxed Code Execution

29 posts · May 2024 – Oct 2026

What this trail covers

Modal's platform supports scaling to millions of concurrent sandboxes, with tens of thousands of sandboxes created per second. This is achieved by rebuilding the core sandbox platform from the ground up, removing central bottlenecks from the control plane, and optimizing container scheduling and startup. The new system utilizes a fleet of horizontally scaled scheduling servers that directly contact workers via RPC, bypassing central datastores in the critical path for sandbox creation. VM Sandboxes can now run alongside Sidecars, which provide a low-latency, isolated security boundary for trusted code, enabling scenarios like custom egress proxies and separating agent harnesses from tool calls. Sidecars share CPU and memory resources with the main Sandbox and can be created dynamically at runtime.

All Modal trails
  1. Runtime Roundup: VM Sandboxes, Multi-node clusters, and more | Modal BlogIntroduces VM Sandboxes, providing agents with full Linux computer access for running Docker stacks, local databases, dev servers, graphical environments, and kernel interaction. Also introduces Sandbox Sidecars, containers running alongside main sandboxes on the same host with the same isolation, enabling trusted operations to run in a Sidecar inaccessible to the agent, with local communication to avoid network round trips.
  2. Sidecars: A low-latency trust boundary for Sandboxes | Modal BlogIntroduced Sidecars, a new isolation primitive that runs alongside a main Sandbox on the same host. Sidecars provide a security boundary between trusted and untrusted code, enabling 3x faster communication (0.58ms p50, 1.4ms p99) compared to separate Sandboxes (2.6ms p50, 44ms p99). This allows for scenarios like injecting credentials, proxying traffic, redacting data, or running harness logic without requiring every operation to cross a remote service boundary. Sidecars are created dynamically via the SDK, share CPU/memory resources with the main Sandbox (up to 250 Sidecars per Sandbox), and have independent outbound network policies. A customer example from Ramp demonstrates using Sidecars to build a custom egress proxy for monitoring and controlling agent outbound traffic.
  3. VM Sandboxes: Full computers for agents | Modal BlogIntroduced VM Sandboxes as a new runtime option for Modal Sandboxes, offering a full Linux VM environment. This is built on Cloud Hypervisor and includes innovations in host filesystems, lazy image loading, memory bursting, and snapshotting. The VM runtime is designed to support agents that require traditional computer capabilities like running Docker stacks, local databases, and dev servers, while maintaining compatibility with existing Modal APIs and performance characteristics like sub-second cold starts and CPU/memory bursting.
  4. Product updates: Sandbox Sidecars, new models, a refreshed dashboard, and more | Modal BlogIntroduced Sandbox Sidecars in public alpha, allowing additional containers to run alongside a main Sandbox on the same host, connected via a low-latency internal network. Provided code examples for using sidecars for custom network proxies and multi-container workloads. Announced VM Sandboxes entering public beta, supporting workloads requiring a real Linux kernel, Docker-in-Docker, eBPF, systemd, cgroups, or custom filesystem mounts. Highlighted improved filesystem performance for I/O-sensitive workloads with VM Sandboxes. Announced Sandbox directory snapshots and the new Sandbox filesystem API are now GA, with directory snapshots allowing snapshots of selected directories for faster session resumptions and state restoration, and the new API offering improved reliability and performance for small files. Shipped new graphs for better observability of Sandbox CPU and memory requests to aid in tuning request sizes.
  5. Devin Outposts on Modal | Modal BlogIntroduced `modal-devin`, an open-source integration enabling Devin AI agents to run within Modal Sandboxes as 'Outposts'. This allows Devin to leverage Modal's features like fast cold-starts, custom images, snapshotting, cost-efficient bursting, and large-scale sandbox deployment. The integration provides a `scheduler` and `session` function for managing agent execution, allowing agents to use custom toolchains, persist their state via workspace snapshots, and access specific hardware like GPUs on demand.
  7. Scaling to 1 million concurrent sandboxes in seconds | Modal BlogThis post details the complete architectural overhaul of Modal's sandbox platform to achieve massive scale. Key changes include replacing the centralized scheduler and durable datastore (Postgres) with a fleet of horizontally scaled scheduling servers and workers acting as their own source of truth, publishing state to Redis streams asynchronously. This eliminates central bottlenecks and global coordination from the critical path of sandbox creation, reducing it to two network hops and a CPU operation. It also addresses O(sandboxes) RPCs by batching control messages and discusses optimizations for Linux kernel `rtnl` lock contention during high-volume container networking setup.
  8. Anthropic integration with Modal brings scalable compute to Claude Science | Modal BlogThis post details the integration of Modal's sandboxed code execution capabilities with Anthropic's Claude Science platform. It highlights how Modal provides scalable, on-demand compute for computationally intensive life sciences workloads, such as protein structure prediction, molecule design, and genome-wide CRISPR knockout screens. Key technical aspects include: fan-out execution across hundreds of Modal containers for parallel processing, per-step GPU access for heterogeneous pipelines, shared storage using Modal Volumes for large datasets and model checkpoints, and reproducible environments managed by Modal Images. The post also references Chai Discovery's use of Modal for similar bursty, hardware-agnostic workloads.
  9. Unpacking sandbox startup latency: why started ≠ ready | Modal BlogThis post introduces Readiness Probes for Modal Sandboxes, a mechanism to define and wait for application-level initialization to complete after container startup. It elaborates on the full sandbox startup lifecycle (Created, Scheduled, Started, Ready, In Use), highlighting the gap between 'Started' and 'Ready' which is often overlooked by benchmarks. The post details how Readiness Probes, implemented as TCP port checks or shell commands, integrate with warm pools to ensure sandboxes are truly ready before being handed out, reducing perceived user latency. It also introduces a new 'ready' event to the sandbox timeline in the dashboard for improved observability.
  10. Product updates: VM Sandboxes, Lower latency routing, RBAC, and more | Modal BlogIntroduced VM Sandboxes as a new runtime for Sandboxes, offering a full Linux kernel for advanced workloads like eBPF and systemd. Added domain allowlisting for Sandboxes to restrict outbound network connections. Introduced Named Images as the recommended way to handle Image builds in Sandboxes, decoupling image builds from app deployment. Sandbox readiness probes are now generally available. Sandbox Filesystem and Directory Snapshots now support configurable TTLs and timeouts.
  11. Introducing Claude Managed Agents with Modal Sandboxes | Modal BlogThis post details the integration of Modal Sandboxes with Anthropic's Claude Managed Agents. It highlights how Modal Sandboxes provide a secure and isolated environment for running tool calls within agentic workflows. The post emphasizes Modal's capabilities such as fast startup with custom images (demonstrated with an example of installing ffmpeg, imagemagick, and mediainfo), persistence options (Volumes and various snapshot types), cost-efficient burst pricing, network security (connect tokens), and scalable compute resources (CPU, memory, GPU). It also includes testimonials from Mason AI, DoorDash, and Blend, showcasing the benefits of this integration for enterprise AI agent development.
  12. Building with Modal and the OpenAI Agents SDK | Modal BlogThis post details the integration of the OpenAI Agents SDK with Modal Sandboxes to build custom agent harnesses. It introduces concepts like `SandboxAgent`, `WorkspaceShellCapability`, `ModalSandboxClientOptions` for GPU attachment, and `SQLiteSession` for stateful interactions. The post also describes advanced patterns like using subagents for higher-order planning and parallelization via a `SubAgentPool` with `asyncio.Future` for non-blocking execution.
  13. Product Updates: RTX Pro 6000 Blackwell, Command K, Sandbox FS API and more | Modal BlogIntroduced a new, more reliable and stable Sandbox Filesystem API (in Beta) that supports reading files up to 5GB, writing files of any size, streaming in both directions, and syncing data to volumes V2. Enhanced CLI for Modal logs with historical fetching (counting or time-based) and filtering by various criteria (search, source, function, container). Added a 'recreate' deployment strategy for immediate container termination upon deployment completion, useful for dev workflows and Apps at max_containers.
  14. Directory Snapshots: Resumable project state for Sandboxes | Modal BlogIntroduced Directory Snapshots, a new feature for Sandboxes that allows specific directories to be snapshotted and mounted independently of the base image. This enables granular state management, separating system dependencies from application code, facilitating pre-warming of Sandboxes with project-specific code, and optimizing file loading by prioritizing snapshot contents. The implementation involves `snapshot_directory` and `mount_image` operations, leveraging Modal's existing lazy-loading container filesystem.
  15. How Ramp built a full context background coding agent on Modal | Modal BlogThis post details how Ramp built a full-context background coding agent ('Inspect') on Modal Sandboxes. Key technical contributions include: leveraging Modal Sandboxes to spin up full development environments (Postgres, Redis, Temporal, RabbitMQ, etc.) in seconds for AI agents; integrating agents with Ramp's stack (GitHub, Slack, Buildkite, Sentry, Datadog) for end-to-end verification; using Modal Functions on a cron job to clone repos, install dependencies, and build filesystem snapshots for near-instant sandbox startup; utilizing Modal Dicts for session locks and image metadata coordination; and employing Modal Queues to route prompts and decouple input from execution for multiplayer collaboration. The focus on matching local agent speed via filesystem snapshots and providing unlimited scale through sandboxes is highlighted.
  16. Product Updates: Modal in AWS & GCP marketplaces, Sandbox improvements, and more | Modal BlogIntroduced UI updates to the Sandbox observability, providing clearer resource and region details, and a new execution timeline showing each Sandbox's full lifecycle from creation to termination.
  17. Product updates: Updates to Volumes, JS and Go SDKs, and moreIntroduced App tags for adding key-value metadata to Apps for better organization and cost tracking. Added Sandbox Connect Tokens for making secure HTTP and WebSocket connections to Sandboxes using authenticated requests. The Modal Vibe demo showcases building a scalable AI coding platform using Sandboxes and Tunnels. Meta FAIR, Cognition, and Restate are using Sandboxes for AI research and agentic coding systems.
  18. Build an AI coding platform that scales to millions of monthly sessionsThis post demonstrates building a scalable AI coding platform ('Modal Vibe') on top of Modal Sandboxes. It details the architecture of a sandbox manager, web frontend, and fleet of sandboxes. The key technical contribution is showcasing how Modal's existing sandbox capabilities (fast startup, high creation rate, parallel execution) enable scaling to thousands of concurrent sandboxes and millions of monthly sessions, with performance metrics provided for scaling from 1 to 1000 apps, highlighting throughput increases and the AI API as a potential bottleneck.
  19. Inside Modal Notebooks: How we built a cloud GPU notebook that boots in secondsThis post details the engineering behind Modal Notebooks, focusing on the systems work that enables fast, interactive GPU workloads. Key contributions include: 1. The use of Modal Sandboxes as the compute layer, enhanced with a lazy-loading content-addressed FUSE filesystem for rapid image startup. 2. The development of `modal-kernelshim`, a daemon within sandboxes that translates Jupyter protocol messages over ZeroMQ to HTTP calls tunneled through the control plane, enabling real-time output streaming. 3. Improvements to scheduling and capacity management to handle idle kernels by automatically pausing and resuming them in seconds. 4. Integration with VolumeFS for persistent, global storage. 5. Implementation of real-time collaboration using Rushlight and Redis Streams, with larger results pushed to S3 Express One Zone. 6. Integration of Language Server Protocol (LSP) with Pyright and auto-formatting with Ruff's WebAssembly build for enhanced editor features. 7. Experimentation with AI completion models like Claude 4 and hosting Zed's Zeta model on Modal GPUs for inference.
  20. Introducing Notebooks – launch ML experiments with zero cold bootsIntroduced Modal Notebooks, a collaborative environment for high-performance interactive computing. Key technical contributions include: a GPU-enabled Python kernel launching in seconds (<5s cold-start) on arbitrary container images and hardware up to 8 H100/B200 GPUs; auto-idling and resume functionality for kernels to optimize cost; seamless integration with existing Modal Apps, including Volumes (distributed storage) and Secrets; true real-time collaborative editing with multiple cursors and live presence; integration with Pyright language server for enhanced developer experience; and AI completions for code exploration. The underlying infrastructure leverages Modal's existing AI infrastructure and FUSE filesystem for fast package loading.
  21. What is an AI code sandbox?This post defines and explains the concept of AI code sandboxes, detailing their historical evolution and current relevance in the context of LLM-generated code. It outlines the technical underpinnings of sandboxes, mentioning gVisor for syscall interception and isolation, and lists key features such as fast cold starts, strong isolation, elastic scaling, and dynamic runtime environments. The post also discusses various use cases for AI code sandboxes, including background coding agents, code reviews, LLM code interpreters, reinforcement learning, and AI-generated apps, positioning Modal as a solution provider for these needs.
  22. Product updates: Multi-node training clusters, B200 and H200s, and Client 1.0 releaseIntroduces multi-node training clusters via the `@clustered` decorator, enabling co-scheduled and co-located GPUs across multiple hosts with high-speed RDMA interconnect (3.2 Tbps Infiniband) for linear scaling. Highlights Quora's use of Modal sandboxes for Poe, achieving up to 1,000 Sandboxes per second and saving engineering time.
  23. How Modal powered 250,000 Lovable app creations in a weekendThis post details how Lovable leveraged Modal's Sandboxed Code Execution capabilities to handle a massive promotional weekend event, successfully powering an estimated 250,000 application creations in 48 hours. Modal ran over 1 million sandboxes, with up to 20,000 concurrent sandboxes at peak, without any operational incidents for Lovable's platform team. The post highlights the reduction in orchestration code from 15,000 lines to 700 lines by using Modal's Tunnels for secure, low-latency communication between sandboxes and backend services. Lovable plans to further leverage Modal Sandbox capabilities like snapshotting for end-user performance gains.
  24. How Quora uses Modal to run thousands of Python sandboxes simultaneouslyThis post details how Quora integrated Modal Sandboxes into their Poe AI chatbot platform to securely execute LLM-generated Python code. It highlights the security, low latency, and reliability requirements met by Modal's sandboxing capabilities, specifically mentioning gVisor for isolation, fast scaling to 1000 sandboxes per second, and networking primitives like Tunnels and IP allowlisting. The post emphasizes that Modal's sandboxes are essentially Modal Functions without the client, leveraging the platform's battle-tested infrastructure. Quora reported saving 2 engineers' worth of infrastructure maintenance time by using Modal.
  25. Modal SDKs for JavaScript and Go (alpha)Introduced alpha SDKs for JavaScript and Go, enabling developers to call deployed Modal Functions and create/interact with sandboxes from server-side JavaScript or Go code. The SDKs support JSON-like and bytes data types for function calls and provide interfaces for sandbox creation, execution, and termination. This expands Modal's accessibility to non-Python environments.
  26. How sync. uses Modal to lipsync 100 hours of video a dayThis post details how sync. uses Modal's parallel execution capabilities to process over 100 hours of video daily for lipsyncing. The process involves splicing videos into scenes, running face detection and translation across multiple T4 GPU containers in parallel, then passing each scene to proprietary LipSync GenAI models on A100 GPUs. Finally, the dubbed scenes are stitched back together. This demonstrates the platform's ability to handle compute-intensive workloads by breaking them down and distributing them across numerous GPU containers, enabling rapid iteration and high throughput.
  27. Modal Sandboxes are generally availableThis post announces the general availability of Modal Sandboxes, a primitive for safely running untrusted code. It details the `exec` API for executing code in arbitrary languages, dynamic image configuration using the `Image` API, and filesystem snapshots for persistence. It highlights customer use cases like SWE-bench for agent benchmarks, Quora Poe for secure code execution, Codegen for large-scale refactors, and Relevance AI for AI workforce automation. Performance testing demonstrated sandbox creation throughput up to 1000 Sandboxes per second.
  28. Product updates: L40Ss, proxy auth tokens, and sandbox disk snapshottingIntroduced a Filesystem API for reading and writing files within Sandboxes, and disk snapshotting capabilities for Sandboxes. Disk snapshotting allows for creating snapshots of the entire Sandbox state, branching from snapshots, and restoring from snapshots to eliminate cold-start times. Also added `fsnotify-like` file watching and support for larger write payloads (up to 1 GiB) to Sandboxes.
  29. Product updates: Rollbacks, batching, sandbox tunnels & moreIntroduced the ability for `Sandbox` objects to forward TCP ports to the internet, enabling connections to web servers running inside them. Also added the capability to `exec` and interface with additional processes within an existing `Sandbox`. The post explicitly mentions interest in improving Sandboxes for agentic use cases.
  30. Inside the Modal code playgroundThe post details the implementation of an in-browser code playground using Modal's own sandboxing features. It describes how user code is sent to a Modal App, which then spawns a `modal.Sandbox` with the `modal` package installed. The user's code is executed within this sandbox using `modal run`, authenticated with ephemeral token credentials. Standard output and error are streamed back to the frontend, and the sandbox is torn down by default after execution. The example code demonstrates creating a sandbox, mounting a local file containing user code, and streaming its output.
  31. Beat GPT-4o at Python by searching with 100 dumb LLaMAsThis post details the use of Modal Sandboxes to evaluate LLM-generated Python code against the HumanEval dataset. It highlights how Sandboxes provide the necessary secure isolation for executing arbitrary code and running tests, enabling the assessment of LLM performance in code generation tasks. The experiment demonstrates scaling evaluation to thousands of tests in parallel, showcasing the robustness of the Sandboxed Code Execution feature for complex benchmarking scenarios.
  32. Competitive prompt engineeringThis post details the use of Modal Sandboxes for executing untrusted code generated by LLMs in a competitive programming context (Prompt Olympics). It highlights the security benefits of sandboxes in preventing malicious code execution (e.g., `sudo rm -rf /`, zip bombs) and their ability to provide controlled environments with features like GPU acceleration and secrets access for LLM deployment and testing.
  33. Run GPU jobs from Airflow with ModalThis post introduces two methods for integrating Modal with Airflow: calling deployed Modal functions using `Function.from_name` and `lookup`, and executing arbitrary Python code within Modal Sandboxes via a custom `ModalOperator`. The `ModalOperator` serializes Python functions and executes them in a sandbox, allowing for dynamic configuration of resources like GPUs. It also demonstrates passing data between Airflow tasks and Modal functions using Modal Dicts.
  34. How Hunch supercharged AI workflows with Modal SandboxesThis post details how Hunch integrated Modal Sandboxes to safely execute AI-generated code. It highlights the security benefits of isolated and ephemeral containers, the performance gains from Modal's optimizations for dependency installation and low-latency execution, and the scalability provided by Modal's infrastructure. The post includes a simplified Python code example demonstrating the creation of a sandbox, installation of packages, execution of code, and capture of output and generated files. It also showcases user examples of generated Python scripts for Slack webhooks and API data fetching, as well as test-driven development workflows.