Introducing the Together AI Batch API: Process Thousands of LLM Requests at 50% Lower Cost

What this post added

This post introduces the Together AI Batch API, a new system capability for processing LLM requests in batches. It details the workflow for preparing, uploading, submitting, and monitoring batch jobs, including sample Python code and input format. Key technical aspects highlighted are the 50,000 requests per batch limit, 100MB file size limit, independent rate limits, and a best-effort 24-hour completion window. The post also outlines best practices for batch sizes, model selection, and error handling.