How to migrate GitLab groups and projects more efficiently
Aug 2, 2023Magdalena Frankiewicz

Summary
Introduced batching for exporting and importing CI/CD pipeline relations to improve migration performance and enable parallelism. This involves splitting CI pipeline data into multiple batches and using concurrent Sidekiq jobs for import. Also added optimizations for exporting CI/CD pipelines by preloading associations and exporting commit notes as a separate relation. Provided guidance on estimating migration duration based on resource types and discussed strategies for efficient migration, including increasing Sidekiq workers and distributing projects across groups.