Blogs›Pinecone›Metadata-Based Data Operations
Pinecone has introduced new bulk data operations that allow users to update, delete, and fetch records directly using metadata filters. This enhances data management scalability and efficiency by enabling targeted operations on large datasets without needing to collect individual record IDs. The operations leverage existing filter syntax, simplifying integration and common data management tasks like bulk updates, data purges, and selective fetches. Pinecone 2.0 introduces metadata filtering for greater control over search results, allowing users to combine vector similarity search with arbitrary metadata filters to provide more relevant results. This provides fine-grained control over vector search results at ultra-low latencies, accepting arbitrary filters on metadata and retrieving exactly the number of nearest-neighbor results that match the filters. The update also includes hybrid storage for up to 10x lower costs by storing a compressed vector index in memory and the original, full-resolution vector index on disk. The new architecture uses Kafka ingestion and Kubernetes orchestration, separating read and write paths and disassociating storage and compute for fault tolerance, data persistence, and high availability. A new REST API based on the OpenAPI spec is available for ease of use, along with a revamped Python client. Pinecone 2.0 is SOC2 Type II audited.
10/30/2025
This post introduces three new bulk data operations: Update by Metadata, Delete by Metadata, and Fetch by Metadata. These operations allow users to target subsets of data using the same filter syntax used in queries, enabling efficient bulk updates, data purges (e.g., for GDPR compliance), and selective fetches. Delete by metadata is GA, update by metadata is in public preview, and fetch by metadata is in early access.
9/14/2021
Pinecone 2.0 introduces metadata filtering for vector similarity search, allowing users to combine vector similarity search with arbitrary metadata filters to provide more relevant results. This provides fine-grained control over vector search results at ultra-low latencies, accepting arbitrary filters on metadata and retrieving exactly the number of nearest-neighbor results that match the filters. The update also includes hybrid storage for up to 10x lower costs by storing a compressed vector index in memory and the original, full-resolution vector index on disk. The new architecture uses Kafka ingestion and Kubernetes orchestration, separating read and write paths and disassociating storage and compute for fault tolerance, data persistence, and high availability. A new REST API based on the OpenAPI spec is available for ease of use, along with a revamped Python client. Pinecone 2.0 is SOC2 Type II audited.