Search Index Reindexing

Notion's search index reindexing process has been significantly optimized. Previously a quarterly, multi-week event requiring constant monitoring and resulting in noticeable gaps, it now completes in under 24 hours with 100% document consistency and zero manual intervention. This was achieved by replacing a custom ECS-based indexing system with an Apache Spark-native pipeline, eliminating a secondary 'catchup' pipeline, and removing external dependencies from the indexing hot path. The new system now uses Spark to generate Elasticsearch native snapshot files directly, bypassing the live indexing API and reducing the operational tax of managing ECS workers.