TechPrufer
Get started
Uber
Uber logoUber · Feature trail

Artifact Storage and Distribution

2 posts · Apr 2026 – May 2026

What this trail covers

Uber has evolved its artifact storage and distribution system from a legacy on-premise solution to a modern, cloud-native approach. The system was modernized by migrating to a managed SaaS offering for artifact storage and implementing an internal proxy layer for validation and cost optimization. This proxy layer reduces egress costs by over 99% by using conditional requests to avoid re-downloading unchanged artifacts, while maintaining comparable latency and reliability. The new architecture provides high-performance gRPC endpoints for OpenSearch, enabling faster ingestion and search by replacing REST/JSON APIs with efficient binary serialization and streaming. This work contributes back to the OpenSearch open-source ecosystem.

All Uber trails
  1. Modernizing Artifact Storage at UberThis post details the migration of Uber's artifact storage from a legacy on-premise system to a managed SaaS offering. A key technical contribution is the introduction of an internal proxy layer that uses conditional HTTP requests (If-None-Match, If-Modified-Since) to validate artifact freshness against the SaaS origin, significantly reducing egress costs by over 99% and achieving 99.99% reliability. The proxy maintains an artifact metadata store (MySQL) and serves unchanged artifacts from an internal cache, ensuring correctness through per-request validation rather than time-based cache expiration. The architecture is active-active across two Uber data centers pinned to nearest cloud regions for low latency and includes GeoDNS failover for resilience.
  2. Accelerating Search and Ingestion with High-Performance gRPC™ in OpenSearch™Introduced native gRPC endpoints to OpenSearch, enabling direct communication with gRPC/Protobuf clients for search and ingestion workloads. Designed a gRPC server module that coexists with the existing REST transport, publishing an SPI for plugin extensibility. Developed an automated pipeline for converting OpenAPI specifications to Protobuf, ensuring API parity and backward compatibility. Integrated gRPC APIs into Uber's OpenSearch Gateway, removing an internal translation layer and reducing latency. Achieved significant performance gains, including a ~60% p99 write latency reduction for M3 metrics and a 20-35% reduction in max indexing delay for the M3 Indexer.