Real-time inference for robots at Physical Intelligence | Modal Blog

What this post added

Introduced a specialized QUIC-based transport over UDP with automatic NAT traversal for real-time robotic inference. This system establishes a persistent, bidirectional channel between the robot runtime and the Modal GPU container, streaming observations and receiving action outputs over the same channel. It handles automatic NAT traversal using STUN discovery and UDP hole punching, coordinated via Modal Dict for rendezvous. This reduces network overhead to ~10-15ms and avoids potential request-response stalls in control loops.