Dynamic Batching
Product updates: Running batch jobs with 1M inputs, ephemeral apps, and a new TensorRT-LLM example

Product updates: Running batch jobs with 1M inputs, ephemeral apps, and a new TensorRT-LLM example

4/17/2025

What this post added

Increased the maximum number of inputs per Modal Function to 1 million, raised the `.spawn()` rate limit, and extended the persistence of `FunctionCall` results to 7 days.

Read the original post ↗