How We Built a Semantic Highlighting Model for RAG Context Pruning and Token Saving

Introduced a new open-source Semantic Highlighting model (zilliz/semantic-highlight-bilingual-v1) based on BGE-M3 Reranker v2. This model performs token-level relevance scoring to identify and highlight semantically relevant sentences within retrieved documents for RAG context pruning. It was trained on over 5 million bilingual samples using a reasoning-based LLM annotation pipeline with Qwen3 8B and vLLM for efficient data generation. The model aims to reduce token costs, improve answer quality, and enhance interpretability in RAG systems.