Introducing Baseten Embeddings Inference: The fastest embeddings solution available

Introduced Baseten Embeddings Inference (BEI), a new inference solution specifically optimized for embedding, reranker, and classifier models. BEI achieves over 2x higher throughput and 10% lower latency compared to previous industry standards, with a focus on high concurrency and rapid single responses. It is designed for memory efficiency, allowing deployment on smaller instance types while delivering superior performance. BEI is plug-and-play and integrates with the TensorRT-LLM Engine Builder, supporting various model types and deployment options (Baseten Cloud, self-hosted, hybrid).