Marketing Campaign Impact Measurement

5 posts · Mar 2021 – Sep 2024

What this trail covers

Grab's internal marketing campaign platform, Trident, has evolved with a powerful campaign builder UI that uses a node tree structure. This UI translates into a set of treatments for event processing, enabling complex campaign logic like counters, delays, and limits. The system stores both the node tree and treatments, along with their mapping, to handle campaign updates and executions. This post details the development of a Marketing Attribution Model (MAM) and the use of hold-out groups to evaluate the business impact of individual campaigns, including statistical significance calculations.

All Grab trails

Sep 2024Unveiling the process: The creation of our powerful campaign builderThis post details the evolution of Grab's Trident marketing campaign platform by introducing a new campaign builder UI. The UI uses a node tree structure to represent complex campaign logic, which is then converted into a set of 'treatments' (event, conditions, actions) for execution. The post explains the implementation of 'counter', 'delay', and 'limit' features, and the algorithm for converting the node tree into treatments. It also covers how campaign updates are handled by comparing old and new treatments.
May 2024How we evaluated the business impact of marketing campaignsThis post introduces a Marketing Attribution Model (MAM) that assigns credit to marketing touchpoints based on relevance and recency to estimate business impact (GMV, revenue). It also details the implementation of a hold-out group strategy for evaluating single campaigns by comparing the performance of users who received a message versus those who did not. Statistical significance is calculated using the jStat library to determine the probability that observed differences are due to randomness, providing campaign owners with an analysis of business impact.
Feb 2024The journey of building a comprehensive attribution platformThis post details the evolution of Grab's attribution platform from limited capabilities to a comprehensive solution using Kappa and Lambda architectures. It describes the technical implementation of an Ads Attribution Engine using Kafka, DDB, S3, an in-house stream processing framework, Redis, ScyllaDB, and RDS. It then details the evolution to a unified engine incorporating promo touchpoints, using Druid for analytics. Finally, it outlines the transition to a full-fledged Lambda architecture with Coban for real-time processing and Spark-based ETL for batch processing, leveraging S3 as a data warehouse. The post also discusses challenges with Lambda architecture and proposes an optimization strategy to separate real-time and batch pipelines.
Oct 2023Scaling marketing for merchants with targeted and intelligent promosThis post introduces Bullseye, an AI-driven system for optimizing marketing campaigns for merchants. It details the problem of globally assigned promos and the need for customized, AI-driven targeting. The architecture includes a UI and backend service interacting with an AI model run via a Chimera Spark job. Key implementation components are the Eater Segments Identifier, Promo Designer (with Offer Content Candidate Generation, Campaign Impact Simulator, and Optimal Promo Selection), Customer Response Model, and Hyper-parameter Selection for optimizing business goals. The post highlights the impact of increased sales and reduced promo spend since its adoption in 2021.
Sep 2023Stepping up marketing for advertisers: Scalable lookalike audienceThis post introduces a scalable lookalike audience platform that addresses issues of long SLA, high cost, and unstable data ingestion in the previous system. The new platform leverages user and audience embeddings for efficient lookalike audience creation and retrieval. Key components include a real-time, in-memory retrieval service with a passenger embedding compression algorithm (reducing memory needs by ~90%) and an embedding-based audience creation and updating pipeline that utilizes shared passenger embeddings and a low-cost neural network for audience embeddings and thresholds. The implementation involves training a user embedding model on billions of user actions, generating audience embeddings by averaging passenger embeddings, and using cosine scores for audience identification. For serving, embeddings are stored in memory, and a compression method based on hash tricks (inspired by Bloom Filters) is used to reduce the 20GB embedding size. The impact includes a 15-minute update window for new campaigns, doubled impressions and clicks, improved CTR, conversion rate, and ROI, and a 98% decrease in lookalike audience generation costs.
Advertisement
Apr 2022Supporting large campaigns at scaleThis post details the development of a batch job service within Grab's real-time automation engine, Trident, to support large-scale marketing campaigns. The service addresses the need for sequential action execution for millions of users by leveraging Kafka for work distribution and introducing batch endpoints in downstream services (Rewards, Messaging) to improve efficiency. Key optimizations include using Kafka partitions with hash keys for load balancing, implementing batch API calls to reduce network latency and database load, sharding Kafka streams by action type and country to enable parallel processing of different campaign types and regions, and making messaging actions asynchronous to avoid blocking reward awarding processes. The post outlines the naive approach, the rationale for choosing Kafka, and the iterative improvements made to meet functional and non-functional requirements.
Mar 2021How Grab Leveraged Performance Marketing Automation to Improve Conversion Rates by 30%This post introduces CARA, an in-house automation tool for managing Google Ads creatives. CARA automates the process of identifying and replacing low-performing creatives with new assets from a library. The tool is designed with principles of performance-driven automation, flexibility through standardized naming conventions, inclusivity for future service launches, and scalability by leveraging Google Ads and YouTube APIs and deploying on AWS serverless compute. An experimental launch showed significant improvements in clickthrough and conversion rates.