How we migrated a live routing system using AI-assisted refactoring | Datadog

What this post added

This post details the migration of Datadog's Stream Router system from a Key-Value (KV) storage model (FoundationDB and RocksDB) to a relational model using PostgreSQL for writes and DuckDB for reads. The migration was driven by scaling limitations of the KV model, specifically FoundationDB's transaction size limits, which hindered the system's ability to manage complex relationships between routes, sharding strategies, and rules. The new schema introduces explicit foreign key relationships, allowing the database to enforce constraints and simplifying application logic. The migration process was significantly accelerated by AI-assisted refactoring tools (Claude and Cursor) within a test-driven development workflow, focusing on safe evolution of a critical production system.