Caching Without Marshal Part 2: The Path to MessagePack - Shopify

This post details the migration from Ruby's Marshal serialization to MessagePack for caching. It explains the differences between Marshal and MessagePack formats, focusing on MessagePack's generic nature and its support for extension types. The post describes how Shopify implemented custom extension types for Date, Symbol, Time, DateTime, BigDecimal, and ActiveRecord::Base. For ActiveRecord objects, a custom serializer (`ActiveRecordCoder`) was developed to handle attributes and associations, including circular references, which Marshal handles automatically but MessagePack does not. A phased migration strategy was employed, running both formats in parallel for six months, using a version byte prefix to differentiate payloads and logging unencodable objects to identify necessary extension types and problematic cache usage.