Integrated Inference for Embeddings

10/9/2025

This post details Pinecone's integrated inference capabilities for generating vector embeddings. It explains the traditional embedding process and contrasts it with Pinecone's integrated approach, which collapses multiple steps into a single API call. The post highlights a customer issue where the `upsert_records()` method's automatic inclusion of the full text field in metadata exceeded the 40KB limit. A workaround is provided by directly calling the Inference API (`pc.inference.embed()`) to generate embeddings and then upserting them, offering greater control over metadata.

3/12/2025

This post details the technical optimizations for Pinecone's retrieval inference system, focusing on embedding generation and reranking. It covers model optimization using NVIDIA TensorRT (kernel tuning, layer fusion, dynamic tensor optimization, precision reduction) with performance gains demonstrated for the bge-reranker-v2-m3 model. It also explains the use of NVIDIA Triton Inference Server with dynamic batching for maximizing GPU utilization, customized for different workloads (query vs. passage). The architecture of scaling with dedicated query and passage infrastructure is described, showing how it maintains stable query latencies even under load. Finally, it highlights the simplification of RAG and agent-based applications through integrated inference, reducing API calls and complexity.

12/2/2024

This post announces the General Availability (GA) of integrated inference capabilities for embeddings and reranking, making them accessible via new database endpoints. It introduces new Pinecone models (pinecone-rerank-v0, pinecone-sparse-english-v0) and a Cohere model (cohere-rerank-v3.5) that are fully hosted and managed by Pinecone. The post details how these integrated capabilities simplify AI development by unifying inference, retrieval, and database management, offering seamless scaling and secure networking. It also provides technical details and code examples for using the new features, including creating an index for a specific embedding model using the `create_index_for_model` endpoint and upserting records with the `/records/upsert` endpoint. The post highlights the benefits of combining sparse and dense embeddings for improved search performance and explains the role of reranking in enhancing accuracy and reducing token waste. It also mentions the availability of a new sparse index type in early access.

12/2/2024

Introduces pinecone-rerank-v0, a new reranking model available via the Pinecone inference endpoint. This model uses a cross-encoder architecture to improve relevance scores for query-document pairs, with a maximum context length of 512 tokens. The post details evaluation results against industry-leading models on benchmarks like BEIR and TREC, demonstrating significant improvements in NDCG@10 and MRR@10. It also provides a Python code example for using the reranker.

9/3/2024

This post announces the general availability of Pinecone serverless on Azure and GCP, expanding the platform's reach. It also introduces the public preview of Pinecone Assistant, a new capability for deploying customized assistants. Furthermore, reranking functionality has been integrated into Pinecone Inference, allowing for embedding, reranking, and querying within a single API, aiming to improve RAG application accuracy and reduce latency/cost. Backups are now available for Standard and Enterprise users, enhancing data security and compliance. Additionally, API Key Roles are in early access, providing granular control over data and control plane access.

8/15/2024

This post introduces reranking capabilities to Pinecone Inference. It details how reranking improves AI application accuracy by scoring documents for relevance to a query, reducing hallucination and cost by filtering irrelevant documents before LLM generation. The post explains the typical RAG pipeline stages and quantifies the potential cost savings (up to 85% with gpt4-o) and accuracy improvements. It also highlights the simplification of the AI development stack by integrating embedding and reranking models with the vector database via a single API. A Python SDK example demonstrates how to use the reranking functionality.

8/1/2024

Introduced Pinecone Inference API in public preview, providing low-latency access to embedding and reranking models hosted on Pinecone's infrastructure. This simplifies AI workflows by reducing the need for external tools and infrastructure management. The initial offering includes the multilingual-e5-large model. The post also mentions optimizations using NVIDIA TensorRT and Triton dynamic batching for these inference capabilities.

7/9/2024

This post introduces Pinecone Inference, a new API that provides easy and low-latency access to embedding and reranking models hosted on Pinecone's infrastructure. It highlights the `multilingual-e5-large` model as the initial offering and demonstrates its usage with a Python code snippet for generating embeddings. The post also mentions the flexibility to use other embedding models and providers, and the upcoming addition of more reranking models.