How Reducto improved enterprise-scale document processing latency by 3x

Reducto achieved a 3x reduction in P90 latency for their enterprise-scale document processing workloads by migrating over 30 models to Modal. Key contributions include leveraging GPU memory snapshotting to reduce cold boot times by 83% (from ~70s to ~12s), enabling independent scaling of models by bundling them into single Modal Functions or keeping them separate, and using parameterized Functions to create independent autoscaling compute pools per customer. They also utilized granular regional control to pin latency-sensitive Functions to specific regions.