Inside Modal Notebooks: How we built a cloud GPU notebook that boots in seconds

This post details the engineering behind Modal Notebooks, focusing on the systems work that enables fast, interactive GPU workloads. Key contributions include: 1. The use of Modal Sandboxes as the compute layer, enhanced with a lazy-loading content-addressed FUSE filesystem for rapid image startup. 2. The development of `modal-kernelshim`, a daemon within sandboxes that translates Jupyter protocol messages over ZeroMQ to HTTP calls tunneled through the control plane, enabling real-time output streaming. 3. Improvements to scheduling and capacity management to handle idle kernels by automatically pausing and resuming them in seconds. 4. Integration with VolumeFS for persistent, global storage. 5. Implementation of real-time collaboration using Rushlight and Redis Streams, with larger results pushed to S3 Express One Zone. 6. Integration of Language Server Protocol (LSP) with Pyright and auto-formatting with Ruff's WebAssembly build for enhanced editor features. 7. Experimentation with AI completion models like Claude 4 and hosting Zed's Zeta model on Modal GPUs for inference.