Cerebras

This post details Rox's implementation of dynamic model routing to optimize AI inference latency for agentic workflows. It highlights the use of Cerebras Inference for the 'fast path' which prioritizes low latency and high tokens/sec for real-time user interactions like chat and voice. The post also discusses how Cerebras is used for short, frequent model calls within orchestration loops to maintain flow and reduce overall user-perceived wait times. The availability of Cerebras Inference through AWS Marketplace is also noted as a key enabler for scalable adoption.