GitLab

GitLab's data migration process for Advanced Search

Jun 1, 2021Dmitry Gruzd
GitLab's data migration process for Advanced Search
SummaryAI, from the source post

Introduced a new, automated, background migration framework for Elasticsearch data, inspired by Rails DB migrations. This framework uses time-stamped migration files stored in Elasticsearch itself, with a background worker to execute them. It supports features like batched migrations, throttling, pausing indexing, and space requirement checks. This framework has been used to migrate issues, comments, and merge requests, and is crucial for the ongoing separation of document types into their own indices.

More from the Database Migration Debugging and Cross-Database Compatibility trail