How we designed Dropbox’s ATF - an async task framework

This post introduces Dropbox's Async Task Framework (ATF), detailing its design, architecture, and guarantees. It addresses the need for a unified async system to handle diverse tasks at scale. Key technical contributions include the callback-based architecture, use of gRPC and Edgestore, SQS for queuing, and the roles of Frontend, Store Consumer, Controller, and Executor. Guarantees like at-least-once execution and no concurrent execution are highlighted, along with lambda requirements for idempotence and resiliency. The post aims to fill an information gap regarding supersized async service development.