Elasticsearch Reindex Machinery

This post details the evolution of Palantir's Elasticsearch reindex machinery within the document store service. It covers requirements for schema-aware reconstruction, low manual action, automatic inconsistency detection, bounded resource usage, and online reindexing. The implementation uses a shadow index, Elasticsearch index aliases, and a schema-level metadata table for orchestration. The system is designed to be cluster-agnostic, supporting both single-cluster and cross-cluster reindexing. This iteration focuses on optimizing the reindex process for performance and reliability, particularly in handling large datasets and complex schema changes, and introduces mechanisms for more granular control and observability during reindex operations.