Scalability and Large-Scale Systems Engineering
Asynchronous computing at Meta: Overview and learnings

Asynchronous computing at Meta: Overview and learnings

1/31/2023 · Sayak Kundu, Artem Denisov

What this post added

This post details the re-architecture of Meta's asynchronous computing platform ('Async tier') to support five-fold growth and enable integration with multiple event sources. Key changes include decoupling storage reading logic from the transport layer, introducing data-source-agnostic 'schedulers' and 'tailers' for external data sources, and transitioning from a pull-based to a push-based model. This addresses challenges with system complexity and external data source integration, improving efficiency by eliminating data duplication in FOQS and enabling first-class support for various data sources.

Read the original post ↗