GitLab

Store and update namespace statistics in a performant manner

Oct 14, 2019Mayra Cabrera
Store and update namespace statistics in a performant manner
SummaryAI, from the source post

Introduced an asynchronous approach to update namespace storage statistics using Sidekiq jobs and Redis for lease management. This avoids increasing the transaction length of project statistics updates and improves performance for large namespaces. The solution involves creating a `namespace_aggregation_schedules` table, enqueuing Sidekiq jobs for immediate and delayed execution, and using Redis leases to prevent concurrent aggregations. A cron job is also scheduled to process any remaining pending updates.

More from the Performance Optimization trail