Multimodal Semantic Search with Images and Text

What this post added

This post details the implementation of a multimodal semantic search application using Milvus. It showcases the use of the Visualized BGE embedding model to jointly embed text and images into a shared vector space, enabling semantic search across these modalities. The post outlines the indexing process, where product images are embedded and stored in Milvus. It then describes the retrieval step, where a multimodal query (text + image) is used to perform a similarity search. Finally, it details a reranking step using Phi-3 Vision to improve the relevance of the retrieved results, demonstrating a complete pipeline for multimodal search.