Deployment and inference for open source text embedding models

This post introduces BEI (Baseten Embeddings Inference), a specialized runtime for embedding models that offers significantly lower latency and higher throughput compared to previous solutions. It details the process of creating a Truss for an embedding model, configuring BEI in `config.yaml` with specific TensorRT-LLM settings, and deploying it to Baseten. The post also demonstrates running inference using an OpenAI-compatible SDK, highlighting the performance gains achieved with BEI.