How LlamaIndex uses Temporal to scale reliable document orchestration

This post details LlamaIndex's migration from RabbitMQ to Temporal for orchestrating complex document parsing pipelines. It specifically showcases how Temporal Workflows are used to implement a robust concurrency control system (semaphores) to manage shared resources like GPU pools and project-specific job limits. The implementation leverages Temporal's atomic start-workflow operations and durable state to ensure fairness, backpressure, and reliable resource allocation, overcoming limitations of their previous system which required significant custom logic for status tracking, heartbeats, and fairness primitives. The post provides code examples of Temporal Workflows used as locks and how they are invoked to acquire and release permits, demonstrating a practical application of Temporal's durable execution for managing distributed system concurrency.