TechPrufer
Get started
Datadog
Datadog logoDatadog · Feature trail

Multi-tenant Data Replication Platform

4 posts · Jun 2025 – Jun 2026

What this trail covers

Datadog built a managed data replication platform to deliver highly reliable, highly scalable, and flexible data movement across the company. This platform abstracts operational overhead, provides robust monitoring and alerting, and adapts to new use cases. It evolved from addressing performance issues in a shared Postgres database by rerouting search queries to a dedicated search platform with dynamically denormalized data. The provisioning process was automated using Temporal workflows, breaking down monolithic data models into smaller, more manageable relational schemas. This post details a migration of the Stream Router's storage backend from a KV model (FoundationDB/RocksDB) to a relational model (PostgreSQL for writes, DuckDB for reads), leveraging AI-assisted refactoring to accelerate the process and ensure live traffic continuity. The new architecture supports complex relational queries and avoids transaction size limits.

All Datadog trails
  1. How we migrated a live routing system using AI-assisted refactoring | DatadogMigrated the Stream Router's storage backend from a KV model (FoundationDB/RocksDB) to a relational model (PostgreSQL for writes, DuckDB for reads). This involved a complete schema redesign to properly reflect relationships between routes, streams, sharding strategies, and rules. AI tools (Claude, Cursor) were used to accelerate the test-driven refactoring process, generating code passes that were validated by failing tests. The new architecture supports complex relational queries and avoids previous transaction size limits, ensuring live production traffic continuity during the migration.
  2. When failover isn’t safe: Building high-availability PostgreSQL on Kubernetes | DatadogThis post details the re-architecture of Datadog's Kubernetes-based PostgreSQL clusters to ensure safe and automatic failover. The previous architecture, which separated leader and read replica pools and used asynchronous replication for standbys, prioritized availability over durability during network partitions, leading to stale data and unsafe failover scenarios. The new architecture leverages synchronous replication for failover candidates, coordinated by Patroni and ZooKeeper, to guarantee data consistency during failover. This involves reconfiguring Patroni to use synchronous replication for standby nodes that are candidates for promotion, ensuring that a standby is sufficiently up-to-date before it can be promoted. The post also discusses the trade-offs between durability and latency, and the validation of the approach through benchmarking and failure testing.
  3. Replication redefined: How we built a low-latency, multi-tenant data replication platform | DatadogThis post details the evolution of Datadog's data replication capabilities, focusing on the development of a managed, multi-tenant data replication platform. It describes the initial challenge of scaling a shared Postgres database for search workloads, leading to the re-architecture of routing search queries to a dedicated search platform with dynamically denormalized data, achieving up to 97% reduction in page load times. The platform's provisioning process was automated using Temporal workflows, breaking down complex setup into modular, reliable tasks. The architectural choice of asynchronous replication is highlighted for achieving low latency and high throughput, enabling rapid data freshness for downstream systems while maintaining observability and operational resilience.
  4. Breaking up a monolith: How we’re unwinding a shared database at scale | DatadogThis post details Datadog's strategy and execution for unwinding a large, shared PostgreSQL database into independently owned instances. It outlines the challenges of shared databases (maintenance coordination, noisy neighbors, brittle schema, access control limitations) and the platform investments (Rapid framework for API/gRPC services, OrgStore managed Postgres platform, PG Proxy for connection layer abstraction) that enabled this migration. The migration process is described in three phases: logical separation (identifying data domains via DBM query samples and team collaboration), access reduction (enforcing domain boundaries by making teams use services instead of direct DB queries), and physical separation (executing the split with minimal risk).