Shepherd: How Stripe adapted Chronon to scale ML feature development
4/15/2024
This post introduces Shepherd, a system built at Stripe that adapted the Chronon framework to scale ML feature development. It details the technical challenges of managing ML features at Stripe's scale, including data versioning, feature lineage, and efficient feature computation. Shepherd's architecture and implementation are discussed, highlighting how it integrates with existing ML infrastructure to streamline the feature development lifecycle from experimentation to production.