How we store and process millions of orders daily

Designed and implemented a dual-database architecture for the Grab Order Platform, separating OLTP (DynamoDB) and OLAP (MySQL RDS) workloads. Optimized DynamoDB usage with primary keys, Global Secondary Indexes (GSIs) with sparse indexing for ongoing orders, and a phased TTL implementation for data retention. Utilized MySQL partitioning for OLAP data retention. Established a Kafka-based data ingestion pipeline with SQS for producer failure handling and back-off retries for consumer consistency, addressing duplicate and out-of-order messages with unique keys and version updates.