Real-time inference for robots at Physical Intelligence | Modal Blog

This post details how Physical Intelligence (Pi) built a specialized QUIC-based portal running over UDP with automatic NAT traversal for real-time robotic inference. This system establishes a persistent, bidirectional channel between the robot runtime and Modal GPU containers, streaming observations and receiving action outputs over a single channel. It utilizes Modal Dict for rendezvous and STUN discovery for NAT traversal, enabling direct robot-to-container connections without custom relay infrastructure. The implementation in Rust provides high throughput and minimal latency, adding approximately 10-15 ms of network overhead. Additionally, Pi leverages Modal Volumes for fast checkpoint loading (under 30 seconds) and region-pinning for inference deployments to minimize round-trip latency.