TechPrufer
Get started
Pinecone
Pinecone logoPinecone · Feature trail

Integrated Inference for Embeddings

8 posts · Jul 2024 – Oct 2025

What this trail covers

Pinecone Inference provides integrated embedding generation and reranking capabilities, allowing users to generate vector embeddings and perform reranking directly within the platform. This simplifies embedding pipelines by enabling automatic embedding and reranking generation as part of the upsert or query process. The reranking feature supports specific models like bge-reranker-v2-m3 and cohere-rerank-3.5, with plans for more models. This integration aims to reduce infrastructure management.

All Pinecone trails
  1. Simplifying Vector Embeddings with Pinecone Integrated Inference CapabilitiesThis post details a workaround for the 40KB metadata limit when using Pinecone's `upsert_records` method with integrated inference. The issue arises because `upsert_records` automatically includes the full text field in metadata. The solution involves directly calling the Pinecone Inference API to generate embeddings, then upserting these embeddings with controlled metadata, thus avoiding the automatic inclusion of the full text and circumventing the metadata size constraint.
  2. Retrieval Inference for scale and performanceThis post details Pinecone's optimizations for retrieval inference, focusing on embedding generation and reranking. It covers model optimization using NVIDIA TensorRT (kernel tuning, layer fusion, dynamic tensor optimization, precision reduction), hardware selection, and maximizing GPU utilization with NVIDIA Triton Inference Server's dynamic batching. The post also highlights the scaling strategy of using dedicated query and passage infrastructure to ensure low-latency query responses are not impacted by batch passage embedding operations. It emphasizes how these integrated inference capabilities simplify RAG and agent-based application development by reducing the number of API calls and external dependencies.
  3. Introducing integrated inference: Embed, rerank, and retrieve your data with a single APIThis post announces the General Availability (GA) of integrated inference capabilities within Pinecone, enabling users to embed, rerank, and query data through a single API. It introduces new Pinecone-developed models (pinecone-rerank-v0, pinecone-sparse-english-v0) and Cohere's rerank-v3.5, accessible via new database endpoints. The post highlights simplified integration, seamless scaling with token-based pricing, and enhanced security features like Private Endpoints, CMEK, and RBAC. It provides detailed Python SDK code examples for data preparation, index creation using the `create_index_for_model` endpoint with specified embedding models and field mapping, record upsertion using `upsert_records`, and querying with integrated reranking via `records/search`.
  4. Introducing Pinecone Rerank V0Introduced `pinecone-rerank-v0`, a new cross-encoder model for reranking search results in RAG systems. The model was evaluated against several industry-leading rerankers (Cohere, Voyage AI, Jina AI, BAAI, Google) on BEIR, TREC Deep Learning, Financebench-RAG, and Pinecone-RAG datasets, demonstrating superior performance in NDCG@10 and MRR@10 metrics. The post details the evaluation methodology, including datasets and metrics used, and provides a Python code example for integrating the new reranker via the Pinecone inference API.
  5. Introducing reranking to Pinecone Inference to simplify building accurate AIThis post announces the public preview of reranking capabilities within Pinecone Inference. It details how reranking improves AI application accuracy by refining retrieved documents before they are passed to an LLM, reducing hallucinations and costs. The post highlights the integration of reranking models (starting with bge-reranker-v2-m3) alongside embedding and vector database functionalities within a single API, simplifying the AI development stack. It provides a Python SDK code example for using the reranking API and outlines the pricing model.
  7. Introducing Pinecone Inference to streamline your AI workflowThis post introduces Pinecone Inference, a new API for low-latency access to embedding and reranking models hosted on Pinecone's infrastructure. It highlights the `multilingual-e5-large` model as the initial offering, detailing its characteristics and providing a Python code example for generating embeddings. The post emphasizes the benefit of a single API for embedding, managing, and querying vector data, reducing complexity for AI application development. It also mentions pricing details and future plans for additional models and reranking capabilities.