Improving storage efficiency in Magic Pocket, our immutable blob store

This post details the development and implementation of new strategies to address storage fragmentation in Dropbox's Magic Pocket immutable blob store, specifically targeting severely under-filled volumes caused by a new service. The new strategies include: 1. Enhancements to steady-state compaction (L1) to improve efficiency. 2. A new 'donor-less' compaction strategy that consolidates live data from under-filled volumes into new, densely packed volumes without relying on existing host volumes. 3. A targeted cleanup process for the sparsest volumes, which involves migrating live data to new volumes and retiring the old ones. These changes successfully reduced storage overhead by 15% and improved effective replication.