TechPrufer
Get started
Milvus
Milvus logoMilvus · Feature trail

LLM Model Comparison and Integration

2 posts · Oct 2024 – Apr 2026

What this trail covers

Milvus continues to evolve its capabilities by integrating with and comparing various large language models. This post details Matryoshka embeddings, a technique for creating multi-scale representations within a single vector, enabling efficient similarity searches by leveraging smaller subsets of dimensions for initial searches and progressively larger subsets for reranking. This approach allows for a balance between precision and computational cost, and Milvus seamlessly supports models trained with this technique.

All Milvus trails
  1. DeepSeek V4 vs GPT-5.5 vs Qwen3.6: Which Model Should You Use?This post introduces a new capability thread by comparing DeepSeek V4, GPT-5.5, and Qwen3.6-35B-A3B across practical use cases like real-time information retrieval and concurrency bug debugging. It details the models' specifications, deployment models, and performance in these specific tests. Crucially, it demonstrates how to connect DeepSeek V4 to the Milvus vector database to enhance retrieval-heavy systems and RAG pipelines, highlighting the practical benefits of integrating LLMs with vector databases.
  2. We Trained and Open-Sourced a Bilingual Semantic Highlighting Model for Production RAG and AI SearchThis post introduces a new bilingual semantic highlighting model trained and open-sourced by Milvus. It addresses the limitations of existing keyword-based and semantic highlighting methods in production AI workloads, particularly for RAG and AI search. The model is designed for high precision, low latency, multilingual coverage, and robustness, with a permissive MIT license for commercial use. It also provides a comparative analysis of existing semantic highlighting solutions like OpenSearch Semantic Highlighter, Provence/X Provence, and Open Provence, highlighting their respective drawbacks.
  3. How to Choose the Right Embedding Model?This post introduces a six-step framework for selecting the right embedding model: 1. Define Task and Business Requirements (task type, ROI, constraints), 2. Evaluate Data (modality, domain specificity, embedding type), 3. Research Available Models (popularity, copyright/licensing), 4. Evaluate Candidate Models (evaluation metrics like faithfulness, answer relevancy, context precision, recall; benchmark performance; load testing), 5. Model Integration (weights selection, self-hosting vs. third-party, API design, caching, batch processing, vector database selection), and 6. End-to-End Testing (performance metrics like MRR, MAP, NDCG, precision, recall, F1, throughput, latency; robustness). It emphasizes matching models to specific use cases and data characteristics.
  4. Multimodal Semantic Search with Images and TextThis post details the construction of a multimodal semantic search application using Milvus, HuggingFace models (Visualized BGE for embedding, Phi-3 Vision for reranking), and an Amazon reviews dataset. It showcases how a multimodal embedding model can jointly embed text and images into the same space, enabling semantic understanding of user intent. The retrieval step performs a similarity search, and a reranking step using an LLVM improves the relevance of results. A key technical takeaway is the ability of the embedding model to relate open-ended text instructions to query images.
  5. Matryoshka Embeddings: Detail at Multiple ScalesThis post introduces Matryoshka embeddings, a novel embedding technique that embeds multiple scales of representation within a single vector. It explains the "funnel search" approach for accelerating similarity searches by iteratively reranking candidates using progressively larger subsets of embedding dimensions. The post also details the modified training objective (Matryoshka loss) and lists popular Matryoshka embedding models that are compatible with Milvus.