Pinecone

Introducing import from object storage for more efficient data transfer to Pinecone serverless

Sep 24, 2024Ben Esh, Gibbs Cullen
Introducing import from object storage for more efficient data transfer to Pinecone serverless
SummaryAI, from the source post

Introduced a new capability for bulk importing data directly from object storage (initially S3, with GCS and Azure Blob Storage planned) into Pinecone serverless namespaces. This feature offers up to 6x lower ingest costs compared to traditional upsert operations, handles ingestion of billions of records asynchronously, and supports Parquet source files. It is designed to streamline large-scale data ingestion for POCs, tenant onboarding, model experimentation, and production workload migrations.

More from the AI Application Performance Measurement trail