Oxidizing Cross-Encoders - Qdrant

This post details the "oxidation" of cross-encoder inference into a Rust library, cross-encode-rs. It explains the underlying concepts of cross-encoders and ONNX Runtime, outlines the Rust implementation for tokenization, batching, and inference, and presents benchmarks comparing its performance against Python libraries (fastembed and sentence-transformers) using ONNX Runtime. The benchmarks show that cross-encode-rs offers competitive or improved inference latency, especially for larger models, and comparable model load times.