Facebook Chat

3/6/2024

This post details the technical implementation of messaging interoperability for WhatsApp and Messenger to comply with the EU's Digital Markets Act (DMA). It describes the use of the Signal Protocol for end-to-end encryption, the construction of message protobuf structures encrypted and packaged into XML stanzas, and the client/server architecture for interoperation. Key technical aspects include the use of XMPP for client-to-server communication, HTTP for interfacing with third-party servers, OpenID and JWT for user identifier authentication, and the Noise Protocol Framework for client-to-server encryption. It also discusses the trade-offs and challenges of not having direct control over both sending and receiving clients, and explores an alternative proxy-based approach.

9/7/2023

This post details the rapid development and launch of Meta's newest social app, Threads. It highlights the startup-like environment, the use of existing technologies like Python (Django) and Instagram's infrastructure, and the ambitious goal of creating a decentralized and interoperable social platform. The post emphasizes the speed of development, the successful scaling to over 100 million users, and the future vision of integrating with protocols like ActivityPub to achieve email-like interoperability.

7/26/2022

This post details the engineering effort to launch Instagram Messaging on desktop web. Key technical contributions include: architecting the mobile web messaging experience with purely functional React components, moving from GraphQL to RESTful endpoints, implementing MQTT for real-time communication, using encrypted push notifications, expanding the ServiceWorker framework, and creating an isolated Redux store and client-side caching architecture. The desktop version reused mobile web code, adapted to larger interfaces with features like drag-and-drop, presence, voice messages, and improved caching.

7/14/2021

This post details the technical architecture and security considerations for WhatsApp's new multi-device capability. It explains how WhatsApp moved from a smartphone-centric architecture to a standalone multi-device experience, addressing challenges in maintaining end-to-end encryption, synchronizing data across devices, and adapting voice/video protocols. Key technical contributions include the client-fanout approach for message delivery, new technologies for device identity management and verification, and methods for synchronizing message history and application states while preserving privacy.

12/21/2020

Introduced 'rsys', a new, smaller, and faster cross-platform video calling library for Meta's apps. Rewritten from scratch using the latest WebRTC, rsys is approximately 20% smaller than the previous library, supports multiple platforms (Android, iOS, MacOS, Windows, Linux), and features a modular architecture with pluggable components and a Flux-based SDK for extensibility. Key technical improvements include a unified state machine architecture for signaling and media stacks, abstraction of OS-dependent functionality, and optimization of binary size through selective compilation and use of optimally sized libraries.

12/16/2020

This post details the development of 'co-experiences', specifically focusing on the 'Watch Together' feature. It introduces a new state synchronization platform ('state sync') built on top of existing multiway infrastructure to handle real-time synchronization of playback controls and audio streams across devices for synchronous viewing. The platform uses Thrift for defining message structures and operates in three parts: subscription, input (snapshot-based model), and resolution (centralized server-side resolver). This platform is designed to be generic and is already powering other features like screen-sharing moderation in Messenger Rooms and Rooms to Live, and supports cross-app communication.

3/2/2020

This post details Project LightSpeed, a massive undertaking to rewrite the Messenger iOS codebase from scratch. It highlights the achievement of making the app twice as fast and one-fourth the size by reducing core code by 84%. Key technical contributions include leveraging native OS features for rendering and threading, using dynamic UI templates powered by SQLite for code reuse, and implementing a server broker for efficient communication. This represents a significant architectural shift to improve performance, reduce complexity, and ensure long-term scalability for Messenger.

10/9/2018

This post details the development of Safety Check for Workplace, an enterprise crisis response tool derived from Facebook's public Safety Check feature. Key technical contributions include migrating storage to TAO for scalability, implementing a custom access management module for enterprise privacy, developing a location parser to standardize company location data, utilizing asynchronous jobs for parallel employee location and notification, fanning out async jobs across batches for timely delivery, and using a queuing system for handling high volumes of responses while updating statuses in real-time via GraphQL subscriptions. The system leverages React, Relay, GraphQL, and Hack, and includes programmatic simulations for resiliency testing.

6/26/2018

This post details the significant overhaul and modernization of Messenger's storage service to improve performance, efficiency, reliability, and ease of upgrades. Key contributions include the migration from HBase to MyRocks (integrating RocksDB as a MySQL storage engine) and from spinning disks to flash storage on Lightning Server SKUs. It also elaborates on the complex, two-phase migration process (normal and buffered flows) designed to handle over a billion users without disruption, including data schema redesign, invariant enforcement, and extensive data validation. The post highlights the resulting 90% reduction in storage consumption, 50x lower read latency, and enhanced disaster recovery automation.

10/9/2014

Introduced a new mobile-first sync protocol for Messenger, moving from a pull-based model to a push-based snapshot + delta model using MQTT. Replaced JSON with Thrift for message and delta updates, reducing payload size by ~50%. Developed a new server-side service called Iris, which acts as a totally ordered queue of messaging updates, enabling independent synchronization rates for the Messenger app and traditional storage tiers. Implemented a tiered storage model with Iris's memory, backing store, and traditional disk storage.

11/13/2013

This post details the technical efforts to significantly improve the startup time and overall performance of the Messenger app on both iOS and Android. On iOS, a custom wall-clock tracing utility (FBTracer) was developed to identify and eliminate non-essential startup processes. On Android, bytecode manipulation using ASM was employed to optimize the dependency injection framework, achieving a 30% speed gain. Further performance enhancements included view controller reuse and backgrounding sound playback on iOS, and optimizing the contact schema and queries on Android. Automated diff testing for startup timing was implemented to catch regressions.

12/3/2012

This post details the integration of Facebook Messenger into Firefox 17, leveraging HTML5 WebSockets for persistent client-server connections and MQTT for efficient, low-bandwidth communication. It also describes the use of HTML5 Shared Workers to centralize data fetching and synchronize state across different Facebook UI elements within the browser, overcoming challenges of reusing DOM-dependent JavaScript libraries within a DOM-less worker context.

11/9/2012

This post details the technical and organizational evolution of integrating standalone mobile app features (specifically Camera and Messenger) into the core Facebook for iOS application. It highlights the shift to a unified product team model responsible for cross-platform experiences, the use of native code and timed release cycles for faster development and deployment, and the strategic decision-making process for feature integration based on user experience and added value to the core app. It also mentions the technical implementation of integrating Messenger's chat access (left swipe) and Camera's multi-photo selection into the core iOS app.

8/22/2012

The 'mercury project' improved the reliability of Facebook messages and chat by replacing existing implementations with a more robust version. Key technical contributions include adopting a CommonJS-like module system for better JavaScript dependency management, implementing test-oriented development with over 85% test coverage and a focus on unit testing, and embracing functional programming paradigms to build a highly decoupled system capable of fast, incremental loading. This involved replacing OOP patterns with functional ones, modeling domain objects as structured data, and using continuations for asynchronous operations, which simplified caching and allowed initialization and updates to share code paths.

3/8/2012

This post details the development of Facebook Messenger for Windows, a desktop application that delivers notifications and chat messages. It highlights the use of HTML and JavaScript for client-side rendering and application logic, allowing for server-side updates. For reliable message delivery, MQTT was adopted as the real-time push channel, similar to the mobile Messenger apps. The installation process utilized a meta-installer, and updates were managed using Google's Omaha technology.

1/31/2012

This post details the engineering challenges and solutions involved in building Facebook's video calling feature, emphasizing user experience, minimal clicks, small binary size (in collaboration with Skype), and simplified UI. It also touches upon the broader evolution of messaging towards a unified communication platform that abstracts away channel management, allowing users to reach others without considering the specific medium.

9/8/2011

This post details how check-in data, a real-time user interaction feature, was leveraged to develop algorithms for predicting business operating hours. It also describes the implementation of crowd-sourcing tools and validation processes to gather and refine this information, ultimately improving the user experience by making it easier to find open businesses.

8/12/2011

This post details the technical challenges and solutions involved in building Facebook Messenger as a standalone app. Key technical contributions include overcoming long latency issues by building a new mechanism that maintains a persistent connection using the MQTT protocol for efficient mobile messaging, achieving phone-to-phone delivery in hundreds of milliseconds. It also discusses product decisions around seamlessly integrating different communication channels (chat, text, email) and the iterative testing process with colleagues to refine the user experience.

7/6/2011

Introduced video calling functionality, integrating with Skype for authentication. Addressed technical challenges in client-side state management, cross-browser/OS installation flows, and handling asynchronous interactions between the server and the client plugin. Minimized installation clicks and time, and ensured user identity protection by using encrypted user IDs with Skype. Explored workarounds for browser reload behaviors post-plugin installation and managed complex call states.

5/2/2011

Introduced the Send button, a new social plugin that allows users to send links to a subset of friends, differentiating from the Like button's broadcast nature. Implemented a flyout dialog for recipient selection and message composition, featuring a custom typeahead with recipient suggestions, friend/group combinations, and external email support. Extended core Facebook tokenizer and typeahead modules in JavaScript, and developed backend code for recipient suggestions. Leveraged the Messages system for sending to users, groups, and email addresses. Enabled seamless integration with the Like button via an XFBML tag.

4/29/2011

This post details the internal architecture and business logic of the Facebook Messages application server. It describes how the server handles message operations (create, read, delete, update), integrates with other Facebook services, and manages group threads using a chat room model with denormalized schemas. It also explains the caching of user metadata using an LRU cache to reduce HBase queries, synchronization mechanisms for handling concurrent updates, the storage schema in HBase including metadata entities, indexes, and action logs, and the use of Apache Lucene for real-time search indexing. The post also highlights the use of dark launches for testing and validation.

2/7/2011

Introduced 'Live Commenting' feature, requiring a new system to handle real-time comment updates. This involved moving from a polling-based approach to a push-based approach due to scalability concerns. A novel 'write locally, read globally' data strategy was implemented to manage the high rate of viewer-to-content associations, contrasting with traditional 'read locally, write globally' patterns. This system was built to handle 100 million pieces of content and 650,000 comments per minute.

11/15/2010

This post details the technology behind a new version of Facebook Messages that unifies chat, SMS, and email into a real-time conversation. The infrastructure handles over 350 million users and 15 billion messages per month. The team evaluated MySQL, Cassandra, and HBase for storage, ultimately choosing HBase due to its scalability, performance, and simpler consistency model for handling both volatile and long-tail data. They also developed a new application server from scratch to interface with various services like Haystack for attachments and ZooKeeper for user discovery, ensuring a robust and performant real-time messaging system.

6/16/2009

Optimized memory usage in Facebook Chat's channel servers by explicitly discarding HTTP request headers that were not needed for constructing responses, thereby reducing the memory footprint per request and increasing the number of users that could be served per machine.

2/18/2009

This post details specific engineering challenges and solutions for scaling Facebook Chat. It describes memory footprint optimizations in Erlang channel servers by using character arrays instead of linked lists and forcing garbage collection before sleep. It also addresses heap fragmentation issues in C++ chatloggers caused by lexical_cast within loops, resolved by moving allocations outside the loop. For presence servers, it highlights the trade-off of using zlib compression to manage high write traffic. Operational challenges with load balancers causing connection resets were resolved by adding more load balancers. The post also mentions the use of Scribe for log aggregation and fb303/Thrift for metrics export to monitor system health.

6/13/2008

This post provides an update on the Thrift framework, highlighting its significant evolution in its first year as an open-source project. It details the addition of new language support (including Erlang, which was previously adopted for Facebook Chat), protocol enhancements, performance improvements, and the impact of external contributions. The post also discusses ongoing development at Facebook (asynchronous C++ server) and within the community (Ruby mapping improvements, C mapping). The acceptance into the Apache Incubator is presented as a major milestone for Thrift's future growth and integration. This post demonstrates Thrift's increasing maturity and its role as a foundational technology for inter-service communication at Facebook, directly impacting the scalability and development of systems like Facebook Chat.

5/14/2008

This post details the engineering challenges and solutions for building Facebook Chat, focusing on real-time presence notification and messaging. It addresses the inefficiency of naive presence updates and introduces XHR long polling (Comet) for real-time messaging, highlighting the limitations of Apache and the need for a custom solution. The post discusses the use of Erlang for an in-memory, epoll-driven web server for handling long-polled requests and C++ for a message logging subsystem, emphasizing their clustering and partitioning for reliability. It also explains the role of Thrift in enabling cross-language communication between PHP, JavaScript, Erlang, and C++. Finally, it describes the 'dark launch' strategy used to test and scale the system to millions of users before public release.