Product updates: Rollbacks, batching, sandbox tunnels & more

What this post added

Introduced the `@batched` decorator for dynamic batching, which accumulates requests into dynamically-sized batches to trade off latency for throughput and GPU utilization.