Product updates: Rollbacks, batching, sandbox tunnels & more
Sep 6, 2024

Summary
Introduces the `@batched` decorator which accumulates requests into dynamically-sized batches, allowing users to trade off latency for throughput and GPU utilization. Also adds the ability to forward TCP ports from `Sandbox` objects to the internet and `exec` into existing `Sandbox` objects, with a stated interest in improving Sandboxes for agentic use cases.