How Lemon Slice built real-time generative video with Modal and Daily

What this post added

This post details how Lemon Slice built a real-time generative video product using Modal and Daily. Key technical contributions include: utilizing Modal Functions for scalable inference of a 1B parameter video model, achieving autoscaling for GPU containers based on request volume, leveraging memory snapshotting for faster container initialization, and implementing a low-latency architecture for real-time video conversations. This involved co-locating inference containers using region selection and enabling direct communication via Modal's Tunnel feature to minimize latency between containers processing user input and generating video.