How Lemon Slice built real-time generative video with Modal and Daily

This post details how Lemon Slice built a real-time generative video product by leveraging Modal's infrastructure for scalable GPU inference and low-latency communication. Key technical contributions include: 1. Using Modal Functions for REST endpoints and video model inference, enabling autoscaling of GPU containers based on request volume. 2. Implementing a real-time pipeline using Pipecat, Deepgram, Grok, and ElevenLabs, with direct container communication via Modal's Tunnels. 3. Co-locating inference containers in specific regions using Modal's region selection feature to minimize latency. 4. Integrating with Daily for streaming the final video and audio back to the user, achieving end-to-end response latencies of 3-6 seconds.