AI Research and Development · Meta Feature Trail

Sep 2026An Organizational Second Brain: Building an AI That Learns From ExpertsThis post details the development of an AI agent designed to act as an organizational "second brain" by learning from expert feedback without requiring model retraining. Key technical contributions include: a structured, auditable knowledge architecture separating knowledge from reasoning; a self-improvement loop that compiles expert feedback into verified updates; the use of composable procedures called "recipes" to define imperative analytical workflows; and a system for organizing knowledge based on density and usage frequency, splitting it between a curated wiki and supplementary retrieval-augmented generation (RAG). The system also incorporates mechanisms for human control through checkpoints and escalations.

Aug 2026From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta’s Ads RankingThis post introduces a multi-stage architecture for Meta's ads ranking system that decouples heavy offline user modeling from lightweight online ranking tasks. It also details a learning technique based on dense tokenization and target-aware attention for efficient feature interaction learning. The architecture enables LLM-style scaling laws for ads recommendations, with performance improvements following a log-linear relationship with compute. Key scaling levers identified are balanced model shape, multi-stage tunability, sequence composition (diversity over homogeneity), and semantic feature representation. This system has contributed to a cumulative lift of 6% in conversions on Instagram, 3% in conversions on Facebook, and 3.5% in ad clicks on Facebook. It is integrated into Meta's Generative Ads Recommendation Model (GEM).

Aug 2026GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation ModelThis post details Meta's co-design efforts to double the end-to-end (E2E) training efficiency of its LLM-scale ads foundation model, GEM, achieving 20-25% Model FLOPs Utilization (MFU) and scaling training FLOPs 4x in 12 months. Key innovations include compute efficiency through a customized recommendation kernel library (Jagged Flash Attention - JFA, Generalized Dot-Product Attention - GDPA, BlockAttention) and mixed ultra-low precision training (MXFP8 attention and MLP), and scaling efficiency via topology-aware 5D parallelism with SM-free collectives (2D FSDP + Expert Parallelism for dense parameters, Fully Sharded 2D Model Parallelism for sparse parameters) co-designed with Meta's multi-tiered network hierarchy. JFA specifically addresses jagged inputs by eliminating padding waste and optimizing backward passes, achieving 40-140% TFLOPS improvement over previous versions. GDPA unifies and accelerates diverse attention modules, addressing performance gaps in real-world workloads compared to synthetic benchmarks.

Jul 2026Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel OptimizationIntroduced Hierarchical Interest Representation, an upstream representation layer for Meta Ads deep funnel optimization. This system uses transformer-based graph learning with bias-aware attention and self-supervised cross-view distillation to learn multi-hierarchical interest representations across a large graph. It fuses real-world knowledge (multimodal advertiser and product content processed through LLMs) with engagement signals to enrich sparse interactions and generalize to rare entities. The architecture includes an enriched engagement graph (typed, weighted, time-decayed) and a hierarchical encoder with structure encoders (node, position, edge) and bias composition for topology-aware attention kernels. The system aims to reduce dimension, enrich knowledge, provide unified relational representation, and leverage multi-hierarchical granularities.

Jul 2026Meta’s AI Storage Blueprint at ScaleThis post details the evolution of Meta's BLOB-storage architecture to meet the demands of large-scale AI workloads. Key technical contributions include: a unified metadata schema backed by ZippyDB for O(1) lookups, elimination of dataplane proxy with a fat client SDK for direct streaming from storage servers, regional deployment of BLOB-storage colocated with GPUs, implementation of a distributed data cache leveraging spare GPU host memory (reusing Owl subsystem components), and a readplan metadata cache. Protocol optimizations include hedged reads for laggards and dynamic concurrency control on the client SDK to manage egress spikes. The architecture aims to maximize GPU utilization and research velocity by minimizing storage bottlenecks and data movement.

Apr 2026Modernizing the Facebook Groups Search to Unlock the Power of Community KnowledgeThis post details the modernization of Facebook Groups Search by adopting a hybrid retrieval architecture that blends lexical (Unicorn inverted index) and semantic (Faiss vector index powered by a 12-layer, 200-million-parameter model) search pathways. It introduces a Multi-Task Multi-Label (MTML) ranking architecture to jointly optimize for clicks, shares, and comments. The post also describes an automated offline evaluation framework using Llama 3 to assess search result quality, leading to measurable improvements in search engagement.

Apr 2026How Meta Used AI to Map Tribal Knowledge in Large-Scale Data PipelinesThis post details the development of a pre-compute engine using specialized AI agents to systematically map tribal knowledge within Meta's large-scale data processing pipelines. This engine creates concise context files that act as structured navigation guides for AI agents, significantly improving their understanding and efficiency when working with proprietary codebases. The system addresses the challenge of AI agents lacking context in complex, multi-repository, multi-language code by documenting non-obvious patterns and cross-module dependencies, leading to reduced AI tool calls and faster task completion.

Apr 2026KernelEvolve: How Meta’s Ranking Engineer Agent Optimizes AI InfrastructureIntroduces KernelEvolve, an agentic AI system that automates the generation and optimization of low-level hardware kernels for heterogeneous AI accelerators (NVIDIA GPUs, AMD GPUs, MTIA silicon, CPUs). KernelEvolve treats kernel optimization as a search problem, using a job harness to compile, evaluate correctness and performance, and profile hundreds of candidate kernels. This approach compresses weeks of expert engineering time into hours, achieving over 60% inference throughput improvement for the Andromeda Ads model on NVIDIA GPUs and over 25% training throughput improvement on MTIA silicon. It injects platform-specific documentation into LLM context for hardware-agnostic kernel generation and distills successful optimizations into reusable patterns.

Mar 2026Meta Adaptive Ranking Model: Bending the Inference Scaling Curve to Serve LLM-Scale Models for AdsThis post introduces the Meta Adaptive Ranking Model, a significant advancement in scaling AI recommendation systems for ads to LLM-scale complexity. It addresses the inference trilemma by employing intelligent request routing, inference-efficient model scaling (request-centric architecture, Request-Oriented Optimization, Wukong Turbo), deep model-system co-design (selective FP8 quantization, hardware-aware graph and kernel specialization), and a reimagined serving infrastructure (trillion-parameter scale, multi-card GPU serving). This innovation has resulted in a +3% increase in ad conversions and +5% increase in ad click-through rate on Instagram.

Mar 2026AI for American-Produced Cement and ConcreteThis post introduces BOxCrete, a new Bayesian Optimization model for concrete mix design, and releases the foundational data used to develop it. It details the application of Meta's Adaptive Experimentation (Ax) platform, which uses Bayesian optimization, to intelligently navigate concrete formulations. The post highlights real-world impact through partnerships, including a data center project where an AI-optimized mix reached full structural strength 43% faster and reduced cracking risk by nearly 10%. It also mentions the open-sourcing of Meta's AI framework for concrete optimization under the MIT license and its adaptation by enterprise SaaS platforms like Quadrel.

Mar 2026Ranking Engineer Agent (REA): The Autonomous AI Agent Accelerating Meta’s Ads Ranking InnovationIntroduced the Ranking Engineer Agent (REA), an autonomous AI agent designed to drive the end-to-end ML lifecycle for ads ranking models. REA addresses long-horizon, asynchronous workflow autonomy via a hibernate-and-wake mechanism, high-quality/diverse hypothesis generation using a dual-source engine (historical insights database and ML research agent), and resilient execution within real-world constraints through a three-phase planning framework (Validation, Combination, Exploitation). REA's architecture comprises REA Planner and REA Executor, supported by a Skill, Knowledge and Tool System. In its first production rollout, REA achieved 2x model accuracy and 5x engineering output.

Feb 2026RCCLX: Innovating GPU Communications on AMD PlatformsThis post introduces RCCLX, an open-sourced enhanced version of RCCL for AMD platforms, integrated with Torchcomms. It details two new features: Direct Data Access (DDA) for lightweight intra-node collectives and Low Precision Collectives (LP). DDA algorithms (flat and tree) improve AllReduce latency by enabling direct memory access between ranks, showing 10-50% performance improvements on AMD MI300X GPUs for decoding and 10-30% for prefill. LP collectives utilize FP8 quantization for up to 4:1 compression, reducing communication overhead for large message sizes (≥16MB) and yielding ~7% increase in throughput and ~9-10% decrease in latency for E2E inference workloads. The post also mentions the integration of CTran to AMD platforms for GPU-resident collectives and highlights the ease of adaptation for AI models through the Torchcomms API.

Feb 2026Building Prometheus: How Backend Aggregation Enables Gigawatt-Scale AI ClustersThis post details the crucial role of Backend Aggregation (BAG) in building gigawatt-scale AI clusters like Prometheus. It explains how BAG interconnects thousands of GPUs across multiple data centers and regions by acting as a centralized Ethernet-based super spine network layer. The post elaborates on BAG's implementation, including its connection to different network fabrics (DSF and NSF), inter-BAG topologies (planar and spread), hardware and routing strategies (Jericho3 ASICs, eBGP with UCMP), and design considerations for resilience and long cable distances.

Jan 2026Adapting the Facebook Reels RecSys AI Model Based on User FeedbackThis post details the development and integration of the User True Interest Survey (UTIS) model into Facebook Reels' recommendation system. The UTIS model leverages direct user feedback from surveys to improve interest matching beyond traditional engagement signals like likes and watch time. It introduces a lightweight alignment model layer trained on binarized survey responses and existing model predictions. The UTIS score is integrated into both late-stage ranking (LSR) and early-stage ranking (retrieval) to boost relevant content and reconstruct user interest profiles. Offline metrics show significant improvements in accuracy, precision, and recall, while online A/B tests with over 10 million users confirm increased engagement and retention.

Nov 2025Zoomer: Powering AI Performance at Meta’s Scale Through Intelligent Debugging and OptimizationThis post introduces Zoomer, Meta's comprehensive, automated debugging and optimization platform for AI training and inference workloads. Zoomer provides deep performance insights, enabling energy savings, workflow acceleration, and efficiency gains. Its architecture comprises three layers: Infrastructure and Platform (distributed storage, fault-tolerant pipelines), Analytics and Insights Engine (GPU trace analysis, CPU profiling, memory analysis), and Visualization and User Interface (interactive timelines, dashboards). Zoomer automates profiling triggers, captures diverse data streams (GPU metrics, execution traces, host-level data), and performs distributed analysis (straggler detection, bottleneck analysis, critical path analysis). Advanced capabilities include single-click QPS optimization for inference and specialized features for GenAI workloads like LLM Zoomer.

Nov 2025Efficient Optimization With Ax, an Open Platform for Adaptive ExperimentationThis post introduces Ax 1.0, an open-source platform for adaptive experimentation that uses machine learning, specifically Bayesian optimization, to automate and optimize complex, resource-intensive experimentation processes. It highlights Ax's application in various Meta domains, including hyperparameter optimization for AI models, GenAI data mixture discovery, infrastructure tuning, and AR/VR hardware design. The post emphasizes Ax's role in not only finding optimal configurations but also in providing insights into the underlying systems being optimized.

Nov 2025Meta’s Generative Ads Model (GEM): The Central Brain Accelerating Ads Recommendation AI InnovationThis post introduces Meta's Generative Ads Recommendation Model (GEM), a large foundation model for recommendation systems (RecSys) built on an LLM-inspired paradigm. GEM introduces architectural innovations for efficient scaling, including stackable factorization machines with cross-layer attention for non-sequence features, a pyramid-parallel structure for offline sequence modeling, and InterFormer for cross-feature learning that preserves full sequence information. It also employs multi-domain learning to tailor predictions to specific Meta surfaces while leveraging cross-surface insights. Post-training, GEM uses advanced knowledge transfer techniques, including a Student Adapter for refined supervision and representation learning, achieving 2x the effectiveness of standard knowledge distillation. The training infrastructure was re-engineered to deliver a 23x increase in effective training FLOPs using 16x more GPUs, with improved model FLOPS utilization (MFU).

Oct 2025Scaling LLM Inference: Innovations in Tensor Parallelism, Context Parallelism, and Expert ParallelismThis post details innovations in LLM inference parallelism, specifically Tensor Parallelism (TP) with Direct Data Access (DDA) algorithms (flat and tree variants) to reduce allreduce latency, Context Parallelism (CP) for long-context models (e.g., 1M/10M tokens) using 'ring attention' variants (Pass-KV, Pass-Q), and Expert Parallelism (EP) for MoE models with optimizations for all-to-all communication. It also outlines future directions like N-D parallelism and disaggregated prefill/decoding tiers.

Sep 2025LLMs Are the Key to Mutation Testing and Better ComplianceThis post details the development and application of Meta's Automated Compliance Hardening (ACH) tool, which uses Large Language Models (LLMs) to revolutionize software testing and compliance. It explains how LLMs overcome the traditional barriers to mutation testing, enabling scalable, realistic mutant generation and the detection of equivalent mutants. The post highlights ACH's success in generating privacy-relevant tests and its deployment across various Meta platforms, emphasizing the acceleration of developer velocity and the creation of a safer online ecosystem.

Sep 2025Meta’s Infrastructure Evolution and the Advent of AIThis post details Meta's infrastructure evolution driven by the advent of AI, highlighting the shift from scaling traditional software and physical infrastructure to building specialized AI clusters. It explains the emergence of GPUs for personalized recommendations and the subsequent dramatic scaling of compute capacity required for Large Language Models (LLMs). The post also emphasizes Meta's commitment to open source and open standards in hardware and silicon for AI infrastructure.

Sep 2025A New Ranking Framework for Better Notification Quality on InstagramIntroduced a diversity-aware notification ranking framework for Instagram that applies multiplicative penalties to engagement ML model scores. This framework penalizes notifications based on similarity across dimensions like content, author, and product type, using a maximal marginal relevance (MMR) approach and tunable weights (w_i). Future work includes adaptive demotion strategies and LLM integration for semantic similarity.

Aug 2025Creating AI agent solutions for warehouse data access and securityThis post details the development of an agentic system for Meta's data warehouse to enhance data access productivity and security. It introduces user agents and owner agents, each composed of specialized sub-agents, to manage data access requests and security operations. The system leverages the hierarchical structure of the data warehouse for agent communication and implements context and intention management. A key use case, 'partial data preview,' is described, orchestrating context, granular query-level access control, data-access budgets, and rule-based risk management.

Jul 2025Accelerating on-device ML on Meta’s family of apps with ExecuTorchThis post details the adoption and impact of ExecuTorch, Meta's open-source framework for on-device ML inference, across Instagram, WhatsApp, Messenger, and Facebook. It highlights improvements in model performance, latency, and privacy for features like Instagram's Cutouts (using SqueezeSAM), WhatsApp's bandwidth estimation and video quality models, Messenger's language identification and E2EE features, and Facebook's SceneX model. Specific technical benefits mentioned include faster model loading, reduced average inference time, and decreased ANR metrics, alongside enhanced security through fuzzing tests. The post also invites community contributions to ExecuTorch.

Jul 2025How Meta keeps its AI hardware reliableThis post details Meta's methodologies for detecting and mitigating hardware faults, particularly silent data corruptions (SDCs), within its AI infrastructure. It highlights the unique challenges SDCs pose for AI training (e.g., NaN propagation, corrupted gradient variance) and inference workloads, and introduces novel detection mechanisms like Fleetscanner, Ripple, and Hardware Sentinel to ensure the reliability of AI hardware at scale.

Jul 2025Using AI to make lower-carbon, faster-curing concreteThis post details the development and open-sourcing of an AI tool (SustainableConcrete) leveraging Bayesian optimization (BoTorch, Ax) to accelerate the discovery of low-carbon, high-performance concrete mixes. It describes the multi-objective optimization problem, the key ingredients, the AI pipeline workflow involving lab testing and iterative refinement, and the metrics used (compressive strength, curing speed, slump, sustainability proxy). The successful deployment of an AI-designed mix at a data center construction site is highlighted, demonstrating the practical application of AI in materials science and industrial sustainability.

May 2025Journey to 1000 models: Scaling Instagram’s recommendation systemThis post details Instagram's journey to scale its recommendation system to over 1000 ML models. It highlights the infrastructure challenges faced, including model discovery, safe release, and health monitoring. The post introduces solutions like a model registry for a single source of truth, automated model launch tooling to reduce time-to-launch from days to hours, and the operationalization of 'model stability' as a key metric for prediction accuracy. It also elaborates on the model registry's structure, including model types and criticality tiers, and its integration with Meta's configuration and monitoring tools.

Mar 2025Building multimodal AI for Ray-Ban Meta glassesThis post details the application of multimodal AI to Meta's Ray-Ban Meta glasses, enabling them to process and respond to visual and auditory input. It highlights the challenges of integrating AI into wearable devices and references the AnyMAL research model, which handles multiple input modalities. The discussion touches upon the processing of requests from the glasses, handling moving images, and scaling these AI capabilities for a large user base, with a focus on optimization potential and incorporating feedback.

Dec 2024Meta Andromeda: Supercharging Advantage+ automation with the next-gen personalized ads retrieval engineThis post introduces Andromeda, Meta's next-generation personalized ads retrieval engine. It details innovations in ML model architecture (deep neural networks with sublinear inference cost), feature representation, learning algorithms, indexing (hierarchical indexing jointly trained with retrieval models), and inference paradigms. Key technical contributions include leveraging NVIDIA Grace Hopper Superchip and MTIA hardware for improved performance, achieving +6% recall improvement and +8% ads quality improvement on selected segments. The system addresses scalability challenges from exponential ad creative growth (especially from Advantage+ and GenAI) through hierarchical indexing and model elasticity, enabling agile resource allocation. It also highlights optimized GPU operators, deep kernel fusion, advanced software pipelining, and dynamic reconstruction of latent user-ad interaction signals on-the-fly, leading to over 100x improvement in feature extraction latency and throughput compared to previous CPU-based components, and over 3x enhancement in end-to-end model inference QPS.

Nov 2024Sequence learning: A paradigm shift for personalized ads recommendationsThis post details a paradigm shift in Meta's ads recommendation engine, moving from Deep Learning Recommendation Models (DLRMs) with human-engineered features to sequence learning architectures. The core innovation is the introduction of Event-Based Features (EBFs) which capture sequential user engagement and conversion events directly, replacing aggregated sparse features. The system utilizes new sequence learning architectures, including custom transformer models with attention mechanisms (reducing complexity from O(N*N) to O(M*N)), to process these EBFs. Scaling challenges for variable-length event sequences are addressed through hardware codesign innovations like native PyTorch support for jagged tensors, kernel-level GPU optimizations, and a Jagged Flash Attention module. Further scaling involves increasing sequence lengths using multi-precision quantization and value-based sampling, and enriching event semantics with multimodal content embeddings via customized vector quantization. The system has shown improvements in ad prediction accuracy, leading to higher value for advertisers and increased conversions.

Oct 2024Meta’s open AI hardware visionThis post introduces Meta's open AI hardware vision, detailing new initiatives like the Catalina high-powered rack for AI workloads, the expansion of the Grand Teton platform to support AMD accelerators, and the Disaggregated Scheduled Fabric (DSF) for next-generation AI clusters. It highlights the importance of open hardware for scaling AI infrastructure, discussing advancements in power capabilities (140kW for ORv3), liquid cooling, modular design, and the use of open standards like OCP-SAI and FBOSS for networking. The post also emphasizes the collaboration with Microsoft on disaggregated power racks like Mount Diablo.

Oct 2024How open source AI can improve population estimates, sustainable energy, and the delivery of climate change interventionsThis post details the open-sourcing of training data and sample code for Meta's AI-powered population maps. The dataset includes nearly 10 million labels for over 126 gigabytes of satellite imagery, indicating the presence of buildings. This data, collected between 2011-2020, aims to train machine vision models to more accurately identify buildings in various environments. The release is intended to assist researchers in improving population estimates for applications in climate adaptation, public health, disaster response, sustainable energy, and climate-resilient infrastructure.

Aug 2024How PyTorch powers AI training and inferenceThis post details advancements in PyTorch for LLM training and inference. It covers memory-efficient fine-tuning techniques (architectural and algorithmic) for consumer-grade hardware and challenges/solutions for on-device LLM deployment using quantization.

Aug 2024Inside the hardware and co-design of MTIAThis post details the co-design process behind the second generation of Meta's in-house AI training and inference accelerator (MTIA). It covers the integration with the PyTorch software ecosystem and the optimization for Meta's key application model architectures. The post highlights specific silicon features utilized to accelerate Meta's models, demonstrating how MTIA achieves performance, efficiency, and developer experience for launching models into production.

Aug 2024Bringing Llama 3 to lifeThis post details the engineering efforts behind bringing Llama 3 to life, covering data diversity, volume, and freshness for GenAI; training Llama at scale with data center, networking, and software investments; and handling LLM inference with key parallelism techniques for scaling model sizes and context windows, and deploying complex serving paradigms across heterogeneous hardware.

Aug 2024How Meta animates AI-generated images at scaleThis post details the technical optimizations implemented for Meta AI's image animation feature to achieve large-scale, low-latency generation. Key contributions include: halving floating-point precision (float32 to float16/bfloat16) for reduced memory footprint and faster operations; improving temporal-attention expansion by replicating context tensors after cross-attention layers to reduce compute and memory; leveraging DPM-Solver to decrease sampling steps to 15; combining guidance and step distillation to reduce three forward passes per step to one, requiring only eight solver steps with one U-Net pass per step; and implementing PyTorch optimizations like TorchScripting and freezing for graph simplification and constant transformation, later migrating to PyTorch 2.0 for granular component optimization and advanced parallelism. For deployment, a traffic management system was developed to route requests regionally, minimizing network overhead and latency, and a retry-based probing system was implemented to approximate queueing and prevent failures when near capacity.

Jul 2024AI Lab: The secrets to keeping machine learning engineers moving fastThis post introduces AI Lab, a pre-production framework at Meta designed to accelerate ML engineer velocity by focusing on Time to First Batch (TTFB). AI Lab continuously A/B tests common ML workflows to proactively improve TTFB and prevent regressions. It addresses challenges like precious GPU capacity by using shrunk models and CPU-only configurations for testing. The post details how AI Lab facilitated a 2x increase in TTFB improvements from the Python Cinder runtime and how it automatically detected and prevented a TTFB regression caused by asynchronous logging. AI Lab integrates into the development cycle at O(Code Changes) and O(Releases) stages, using t-tests and ROC curves to confirm statistically significant changes before marking them as regressions or improvements.

Jul 2024Meta’s approach to machine learning prediction robustnessThis post details Meta's systematic framework for ensuring prediction robustness in large-scale ML recommendation models used for advertising. It outlines challenges like model stochasticity, frequent updates, and complex interactions. The framework comprises prevention guardrails (e.g., Snapshot Validator reducing model snapshot corruption by 74%), fundamental understanding (e.g., Hawkeye AI debugging toolkit reducing root-cause analysis time by 50%), and intrinsic robustness techniques (e.g., model pruning, quantization). Specific solutions are detailed for model, feature, training data, and calibration robustness. The post also highlights advancements in ML interpretability for root-cause analysis and understanding model behaviors.

Jun 2024Leveraging AI for efficient incident responseThis post details the development and implementation of an AI-assisted root cause analysis system for streamlining system reliability investigations. It introduces a novel approach combining heuristic-based retrieval and LLM-based ranking (using Llama models) to identify potential code changes responsible for incidents. The post highlights the training process involving continued pre-training and supervised fine-tuning of Llama 2 (7B) models with internal data and a dedicated RCA SFT dataset, achieving 42% accuracy in identifying root causes at the creation time of investigations. It also discusses future directions for AI in incident response, emphasizing explainability, confidence measurement, and proactive incident detection.

Jun 2024PVF: A novel metric for understanding AI systems’ vulnerability against SDCs in model parametersIntroduces Parameter Vulnerability Factor (PVF), a novel metric to quantify AI model vulnerability against silent data corruptions (SDCs) in model parameters. PVF is derived from fault injection experiments and measures the probability that a corruption in a specific parameter leads to an incorrect output. The post details PVF's applicability across different models, tasks, and hardware fault models, and its extension to the training phase. It also highlights Dr. DNA, a method for detecting and mitigating SDCs during deep learning model inference, achieving high detection rates and performance recovery with low overhead.

Jun 2024How Meta trains large language models at scaleThis post details Meta's infrastructure innovations for training large language models (LLMs) at scale. It highlights the shift from training many small models to fewer, massive ones, and the resulting challenges in hardware reliability, fast recovery, state preservation, and GPU connectivity. Key infrastructure developments include optimizing hardware (e.g., modifying the Grand Teton platform with higher TDP GPUs and HBM3), data center deployment strategies for high compute density, and network fabric choices (RoCE and InfiniBand). The post elaborates on network optimizations for LLM training, including assigning communication patterns to network topology, implementing topology-aware collective communication algorithms (e.g., recursive doubling/halving), and enhancing network load balancing and routing for fat flows. It also touches upon storage solutions and the importance of PyTorch and efficient scheduling.

Jun 2024Maintaining large-scale AI capacity at MetaThis post details Meta's strategies for maintaining large-scale AI capacity, specifically focusing on GPU training clusters. It highlights the challenges of ensuring capacity guarantees, handling 'bad hosts,' minimizing interruption rates, ensuring rollout safety, and maintaining host consistency in a dynamic environment. The post introduces 'maintenance trains' for cyclic server maintenance and 'gradual rollouts' for software and firmware updates, emphasizing the need for careful testing and vendor collaboration. It also describes the role of OpsPlanner in orchestrating disruptive work and ensuring host consistency, along with safety features like autostop and automatic offboarding of failing upgrades.

Mar 2024Building Meta’s GenAI InfrastructureThis post details the architecture and implementation of Meta's new 24k GPU AI clusters, designed for GenAI workloads like Llama 3 training. It covers hardware (Grand Teton, OpenRack), networking (RoCE with Arista 7800, InfiniBand with NVIDIA Quantum2), storage (Tectonic FUSE, Hammerspace NFS), and performance optimizations (network topology awareness, NCCL tuning, FP8 support, checkpointing). It also highlights efforts in debuggability (desync debug) and PyTorch evolution for large-scale training.

Jan 2024Improving machine learning iteration speed with faster application build and packagingThis post details improvements to the ML/AI development workflow by addressing slow build times and inefficient executable packaging. Key contributions include: implementing non-determinism mitigation in Buck2 and Remote Execution to ensure consistent build outputs, optimizing dependency management by identifying and removing unused code/dependencies (e.g., GPU code, unused Python modules), and introducing the Content Addressable Filesystem (CAF) for incremental packaging and distribution of Python executables. CAF utilizes a CAS daemon, P2P distribution via Owl, and Btrfs filesystem features (compression, COW) to optimize content fetching and deduplication. Future work includes lazy fetching and enforcing uniform revisions.

Jan 2024Lazy is the new fast: How Lazy Imports and Cinder accelerate machine learning at MetaThis post details the adoption of Lazy Imports and the Python Cinder runtime to accelerate machine learning development at Meta. It highlights the challenge of Time to First Batch (TTFB) in ML training and explains how Cinder's transparent, default lazy import mechanism defers all imports until needed, reducing costly runtime imports and streamlining workflows. The post quantifies improvements, noting up to 40% TTFB reduction and 20% faster Jupyter kernel startup times. It also discusses challenges encountered, such as library compatibility issues with PyTorch, Numba, NumPy, and SciPy, the need to balance performance with dependability by disabling lazy imports before the first batch, and the learning curve for developers. Future plans include streamlining onboarding, enhancing debugging tools, and fostering community collaboration.

Dec 2023AI debugging at Meta with HawkEyeThis post introduces HawkEye, a new toolkit developed as part of Meta's Prediction Robustness program. HawkEye provides end-to-end monitoring, observability, and debuggability for ML workflows, significantly reducing the time spent debugging production issues. It enables isolation of top-line product issues to specific model snapshots, prediction anomalies to features, and upstream causes of feature issues. It also aids in diagnosing model snapshots and training data issues, thereby enhancing the robustness and reliability of ML-based products.

Oct 2023How Meta is creating custom silicon for AIThis post details the development of Meta's custom AI silicon, MTIA (Meta Training and Inference Accelerator). It highlights the challenges of designing hardware for rapidly evolving AI software, the advantage of in-house silicon and software teams for co-design, and the significance of MTIA v1 as Meta's first-generation ML accelerator. The post also discusses learnings from the development process, future directions for AI silicon design (addressing memory and inter-chip communication bottlenecks), and the importance of a collaborative, hands-on team culture.

Sep 2023Using Chakra execution traces for benchmarking and network performance optimizationThis post introduces Chakra execution traces, a new open, graph-based representation of AI/ML workload execution designed to address the limitations of traditional AI benchmarking. It details how Chakra traces capture compute, memory, and communication operations, their dependencies, timing, and resource constraints. The post highlights Meta's use of Chakra for benchmarking (via tools like Mystique and PARAM), visualization, and performance optimization, and outlines future plans for enhancing benchmarking capabilities and using AI to generate representative traces. It also emphasizes the importance of industry collaboration through MLCommons for broad adoption.

Sep 2023Arcadia: An end-to-end AI system performance simulatorIntroduced Arcadia, a unified end-to-end AI system performance simulator designed to model the compute, memory, and network performance of AI training clusters. Arcadia takes various inputs including AI system plans, network topology, workload distributions, and hardware specifications to provide insights into AI model and workload performance, enabling data-driven design decisions for AI clusters and infrastructure. It addresses challenges in optimizing large-scale, multi-layered AI systems by providing a common source of truth for performance analysis and joint optimization across application, network, and hardware levels.

Aug 2023Scaling the Instagram Explore recommendations systemThis post details the evolution of Instagram's Explore recommendation system, highlighting the application of advanced machine learning models like Two Towers neural networks and a multi-stage ranking approach (retrieval, first-stage ranking, second-stage ranking, final reranking). It emphasizes the use of caching and pre-computation for scalability and flexibility, and the adaptation of ML techniques for personalized content discovery at a massive scale.

Oct 2022Improving Instagram notification management with machine learning and causal inferenceThis post details the application of causal inference and machine learning to optimize Instagram's daily digest push notifications. It introduces a method to move beyond traditional click-through rate (CTR) models by estimating the incremental value of sending a notification (i.e., the increase in user activeness). Uplift modeling, based on a randomized experiment where notifications were randomly sent or dropped, is used to predict this incremental impact. A neural network-based uplift model is developed, and to stabilize sending rates, online quantile computation transforms raw uplift estimates to a uniform distribution, allowing for decisions based on a fixed threshold 'r'. This approach significantly reduced notification sending volume compared to CTR models without negatively impacting user engagement.

Sep 2022Scaling data ingestion for machine learning training at MetaThis post details the evolution of Meta's data ingestion infrastructure for machine learning training. It introduces a disaggregated Data PreProcessing tier (DPP) that handles data fetching, decryption, feature extraction, tensor conversion, and last-mile transformations. Key optimizations include feature flattening for efficient selective feature reading (2-2.3x efficiency gains), client-side rebatching to support larger batch sizes and improve reader node throughput (20-40% improvement), and the FlatMaps project to optimize memory bandwidth utilization in data reading workloads. These improvements address the mismatch between hardware performance growth and data ingestion demands, and the power constraints in data centers.

Jun 2022Applying federated learning to protect data on mobile devicesThis post introduces a new system architecture and methodology for applying federated learning with differential privacy (FL-DP) to train ML models on mobile devices. It addresses challenges such as label balancing, feature normalization, metrics calculation due to lack of data visibility, slower mobile release cycles, slower training due to federation, and anonymized system logging. The architecture combines mobile devices, trusted execution environments, and conventional back-end servers, and has been validated with an in-house FL library compatible with Meta's apps, showing minimal performance degradation compared to conventional server-trained models.

Aug 2021Open sourcing Winterfell: A STARK prover and verifierOpen-sourced Winterfell, a general-purpose STARK prover and verifier implemented in Rust. This library aims to simplify the use of zero-knowledge proofs (ZKPs) for developers by abstracting away much of the cryptographic complexity. Winterfell leverages the properties of STARKs, such as minimal cryptographic assumptions, transparency, and performance, making it suitable for applications in blockchain privacy and scalability, as well as general computational integrity proofs. The post details the technical advantages of STARKs, the design of Winterfell, its modular crate structure, and provides guidance on using it through tutorials and examples.

Jul 2021Fully Sharded Data Parallel: faster AI training with fewer GPUsIntroduced Fully Sharded Data Parallel (FSDP) as a new tool for more efficient AI model training. FSDP shards model parameters, gradients, and optimizer states across data parallel workers, improving memory efficiency and computational efficiency through communication/computation overlapping. It is implemented in the FairScale library and offers simple APIs as a drop-in replacement for PyTorch's DistributedDataParallel module. The post details the technical mechanism of FSDP, explaining how it decomposes all-reduce operations into reduce-scatter and all-gather, and provides usage examples for language models (fairseq), computer vision models (VISSL), PyTorch Lightning, and direct FairScale integration.

Jun 2021Asicmon: A platform agnostic observability system for AI acceleratorsIntroduced Asicmon, a scalable observability framework for AI accelerators that abstracts custom interfaces and provides a standard interface to internal tools, facilitating load balancing, performance monitoring, and automated health checks. Developed Asimov, a custom specification language to simplify the development and rapid prototyping of new accelerators by generating efficient C++ monitoring modules. Introduced Atrace, an accelerator tracing solution for remote trace collection and detailed inspection of accelerator systems.

Dec 2020On the value of diversified recommendationsThis post introduces the concept of diversified recommendations in AI systems, moving beyond simple predictive accuracy to consider preference broadening and long-term user engagement. It uses toy models (Polya and balancing processes) to illustrate self-reinforcing and self-balancing dynamics. It then details the complexities of real-world user preferences (multidimensional, soft, contextual, dynamic) and proposes practical methods for diversification, including author, media type, and semantic diversity. It also discusses exploring similar semantic nodes in knowledge graphs, maintaining long and short-term preferences, utilizing explore-exploit tradeoff algorithms (epsilon-greedy, UCB), and incorporating qualitative trade-offs and negative guardrails based on user feedback. The ultimate goal is to optimize for long-term user satisfaction by balancing immediate engagement with preference broadening.

Jul 2019EGG: A toolkit for multi-agent language emergence simulationsIntroduced EGG, a toolkit for multi-agent language emergence simulations. EGG allows researchers to create game simulations where two neural network agents develop their own discrete communication systems to solve tasks. It lowers the barrier to entry for studying language emergence by providing a framework for designing and testing new games, supporting deep sequence-to-sequence modeling and reinforcement learning.

May 2019Releasing Pythia for vision and language multimodal AI modelsThis post introduces Pythia, a new deep learning framework for vision and language multimodal AI models. Pythia is built on PyTorch and features a modular, plug-and-play design to facilitate rapid model building, reproduction, and benchmarking. It supports multitasking, distributed training, custom losses, metrics, scheduling, and optimizers, and incorporates elements from winning entries in AI competitions like the VQA Challenge and Vizwiz Challenge. The goal is to accelerate progress in the vision and language domain by increasing model reproducibility and enabling faster prototyping and experimentation.

May 2019F8 2019 Day 2 keynote and session videosThis post highlights the use of PyTorch for accelerating AI research to production deployment in computer vision, NLP, and machine translation. It also details the application of ML for developer productivity tools such as code search, code recommendation, and automatic bug fixing. Additionally, it introduces adaptive experimentation as an AI-enabled testing approach for product, infrastructure, ML model, and marketing campaign optimization.

Apr 2019Open-sourcing Habana back end for GlowThis post introduces an experimental back end for the Glow compiler and runtime project, specifically designed to target Habana's existing hardware accelerators. This is the first back end to customize for various vendors' accelerators, enabling partners to more rapidly design and optimize new silicon products for machine learning (ML). It strengthens the ecosystem for accelerating neural network ML workloads at production scale, helping companies reduce power usage and latencies. The back end is available on GitHub for targeting Habana's inference accelerator cards and future accelerator products.

Apr 2019PyTorch-BigGraph: Faster embeddings of extremely large graphsIntroduced PyTorch-BigGraph (PBG), a new tool for training multi-relation graph embeddings on extremely large graphs (billions of nodes, trillions of edges). PBG shards nodes and edges, performs multi-threaded training on single or multiple machines, and outputs node embeddings. It is built on PyTorch, allowing for easy customization of loss functions and models. PBG is designed to be robust, scalable, and orders of magnitude faster than commonly used embedding software, producing comparable quality embeddings.

Mar 2019Accelerating Facebook’s infrastructure with application-specific hardwareThis post introduces Meta's next-generation hardware platform for AI training, code-named "Zion," and new custom chip designs (ASICs) optimized for AI inference ("Kings Canyon") and video transcoding ("Mount Shasta"). Zion is a large-memory unified training platform utilizing OCP accelerator modules (OAM) and a disaggregated design for independent scaling of memory, compute, and network components. Kings Canyon focuses on scalable inference servers using M.2 modules, single-socket servers, carrier cards, and chassis, with support for INT8 and FP16 precision. The post also details the challenges of large model partitioning for inference and the role of the Glow compiler in abstracting vendor-specific hardware for ML accelerators. For video transcoding, custom ASICs (Mount Shasta) are designed to improve efficiency and enable new functionalities like real-time 4K 60fps streaming by dedicating silicon for decoding, scaling, and encoding stages.

Mar 2019AI blog roundupThis post announces the launch of the new Facebook AI blog and highlights several key AI research and development efforts. It details the AI powering Portal's Smart Camera, focusing on on-device computer vision and expert knowledge integration. It also introduces a new technique for faster neural network verification for safety-critical applications, a challenge for teaching AI intuitive physics concepts via the IntPhys Challenge, and a Lyapunov-based approach for safe reinforcement learning algorithms. Finally, it touches upon the future of deep learning hardware influenced by architectural concepts discussed by Yann LeCun.

Feb 2019FAIR open-sources new ELF OpenGo data set, research, and insightsFAIR open-sourced a retrained model of ELF OpenGo using reinforcement learning, which outperforms previous versions. They also released the training data from 20 million self-play games, 1,500 intermediate models, a Windows executable of the bot, a new data archive and analysis of 87,000 professional Go games, and an interactive visualization tool. This release serves as a benchmark for future AI research in game playing and beyond.

Feb 2019Cross-lingual pretraining sets new state of the art for natural language understandingIntroduced a new cross-lingual pretraining approach using Translation Language Modeling (TLM) and Masked Language Modeling (MLM) objectives. This method concatenates parallel sentences and trains models to infer masked words by attending to text in either or both languages. Achieved state-of-the-art results in unsupervised and supervised machine translation, and cross-lingual text classification, especially for low-resource languages. Released code and pretrained models to the community.

Jan 2019Teaching AI to learn speech the way children doThis post details the ZeroSpeech 2019 challenge, a research initiative focused on teaching AI systems to learn speech using only audio input, without text or phonetic labels. The challenge aims to develop AI that can discover its own discrete 'orthographic' notation for unknown languages and re-synthesize speech. This contributes to research in unsupervised learning for speech synthesis, particularly for low-resource languages, and advances natural language understanding beyond English-centric work.

Jan 2019No training required: Exploring random encoders for sentence classificationThis post introduces a novel baseline for sentence embeddings by exploring random encoders that require no additional training. It demonstrates that much of the performance gain in sentence embeddings comes from pre-trained word representations, and that random parameterizations can match or exceed the performance of established sentence encoders like SkipThought and InferSent. The work also provides recommendations for proper experimental protocol in sentence classification evaluation.

Jan 2019Zero-shot transfer across 93 languages: Open-sourcing enhanced LASER libraryThis post introduces the enhanced LASER (Language-Agnostic SEntence Representations) toolkit, now supporting over 90 languages and 28 alphabets. It achieves massively multilingual sentence representations by embedding all languages in a single shared space, enabling zero-shot transfer of NLP models. Key technical contributions include a shared encoder (five-layer bidirectional LSTM) and decoder, a joint byte-pair encoding (BPE) vocabulary, and training on 223 million sentences of public parallel data. The toolkit achieves state-of-the-art results on cross-lingual natural language inference (XNLI) and parallel corpus mining (BUCC), processing up to 2,000 sentences per second on GPU.

Jan 2019AI year in reviewThis post details advancements in AI learning through semi-supervised and unsupervised training, including new methods for training automatic translation models on unsupervised data and expanding translation services to 24 more languages. It also highlights the use of hashtags for image recognition training and the release of PyTorch 1.0, integrating Caffe2 and ONNX for a streamlined AI development pipeline. Additionally, it introduces Horizon, an open-source platform for applied reinforcement learning in production, Glow for hardware acceleration, and the Big Basin v2 ML-optimized server design. The post also touches on the transition of Oculus Research to Facebook Reality Labs and explorations in AI and AR/VR research.

Dec 2018Open sourcing wav2letter++, the fastest state-of-the-art speech system, and flashlight, an ML library going nativeThis post introduces wav2letter++, a fully convolutional end-to-end speech recognition system that achieves state-of-the-art performance, and flashlight, a native machine learning library for CPU and GPU backends. Wav2letter++ leverages CNNs for acoustic and language modeling, offering an alternative to recurrent architectures. Flashlight is designed for performance and flexibility, enabling fast iteration and reproducibility for AI research and model tuning.

Dec 2018Nevergrad: An open source tool for derivative-free optimizationThis post introduces and open-sources Nevergrad, a Python library for derivative-free optimization. It offers a wide range of algorithms (e.g., differential evolution, CMA-ES, particle swarm optimization) in a standard ask-and-tell framework, enabling AI researchers to easily test and compare different optimizers for ML problems. The library includes benchmarks for evaluating algorithm performance on various problem types (multimodal, noisy, discrete, ill-conditioned) and can be used for hyperparameter tuning in ML models and other scientific applications.

Dec 2018Open-sourcing PyText for faster NLP developmentThis post announces the open-sourcing of PyText, a new NLP modeling framework built on PyTorch. PyText aims to simplify the workflow for NLP development, enabling faster experimentation and easier large-scale deployment. It offers prebuilt model architectures, utilities for text processing, and access to the PyTorch ecosystem. The framework has been used internally for over a billion daily predictions and has reduced development time from weeks/months to days. It addresses the historical trade-off between experimentation-optimized and production-optimized frameworks by providing a unified approach. PyText supports conversion to ONNX and Caffe2 for production deployment, offering a modular design with features like distributed training, multitask learning, portability, prebuilt models, and contextual modeling. It also improves upon DeepText by supporting dynamic graphs and offering faster training with GPU utilization and distributed training.

Dec 2018Facebook contributes to MLPerf, open-sources Mask R-CNN2GoFacebook contributes to the MLPerf initiative by providing reference implementations for ShuffleNet (image classification) and Mask R-CNN2Go (pose estimation) for the Edge Inference category. They are open-sourcing Mask R-CNN2Go, a computer vision model optimized for embedded and mobile devices, enabling object detection, classification, person segmentation, and body pose estimation for real-time inference. Mask R-CNN2Go currently runs on Caffe2 with plans for PyTorch 1.0 support and is used for AR effects like hand tracking.

Dec 2018PyTorch developer ecosystem expands, 1.0 stable release now availableThis post announces the stable 1.0 release of PyTorch, highlighting its new features such as a hybrid front end for eager and graph execution, revamped distributed training, a pure C++ front end, and deep integration with cloud platforms. It emphasizes how PyTorch 1.0 accelerates the AI research-to-production workflow and discusses the growing PyTorch community, educational programs (Udacity, fast.ai), and new projects extending PyTorch (Horovod, PyTorch Geometry, TensorBoardX, Translate). It also details cloud partner integrations (AWS SageMaker Neo, Google Cloud Deep Learning VM, Azure Machine Learning).

Dec 2018FAIR turns five: What we’ve accomplished and where we’re headedThis post details FAIR's five-year progress in AI research, highlighting key contributions: 1. Memory Networks: Developed to address the limitation of neural networks in storing long-term information, enabling them to answer general knowledge questions and refer to previous statements. This evolved into StackRNN, bAbI datasets (now part of ParlAI), end-to-end memory networks, and key-value memory networks. 2. Self-supervised learning and Generative Models: Explored Generative Adversarial Networks (GANs) starting in 2015, publishing papers on DCGANs, LAPGANs, and AGDLs to demonstrate their effectiveness in generating realistic images and video. Introduced Wasserstein GAN (WGAN) in 2017 for more robust and stable adversarial training. 3. Text Classification: Developed fastText in 2016 for rapid text classification and word representation learning, assigning vectors to subword units for scalability and faster training. fastText is now used for text classification across 19 languages at Facebook and is a common baseline for word embedding systems.

Nov 2018Using AI and satellite imagery for disaster insightsIntroduced a framework using CNNs on satellite imagery to identify disaster-affected areas by detecting changes in human-made features (roads, buildings) between pre- and post-disaster snapshots. Developed a Disaster Impact Index (DII) to quantify affected regions, correlating strongly with actual disaster impact in test cases (Hurricane Harvey, Tubbs Fire). This method aims to provide faster and more accurate disaster mapping information to rescue workers.

Nov 2018Neural Egg Separation: Training ML systems to extract audio/video from noisy/cluttered environmentsIntroduced Neural Egg Separation (NES), a semi-supervised iterative method for isolating audio and visual sources in noisy environments. NES combines aspects of supervised training and estimation by mixing signals and conducting multiple analyses, gradually injecting known signals while the model improves at extraction. Experiments show NES outperforms other methods with similar supervision levels and is competitive with fully supervised systems.

Nov 2018New fastMRI open source AI research tools from Facebook and NYU School of MedicineThis post details the release of open-source AI tools and data for the fastMRI project, a collaboration between Facebook AI Research (FAIR) and NYU School of Medicine. It includes new AI models and baselines for accelerating MRI scans, as well as the first large-scale MRI dataset of its kind. The contribution aims to standardize research by providing a common set of tools and data, enabling researchers to compare results on a leaderboard and advance AI-driven image reconstruction techniques for faster and more accessible diagnostic imaging.

Nov 2018Toward a deeper understanding of the way AI agents see thingsThis post details a study on AI agents learning to communicate about images. The research found that agents relied on low-level feature similarities rather than conceptual understanding of image content. This finding is significant for designing future AI systems that aim for more natural communication strategies by understanding the visual semantics agents use.

Nov 2018Making floating point math highly efficient for AI hardwareThis post introduces a novel approach to optimizing floating-point arithmetic for AI hardware, aiming to improve efficiency and accuracy compared to traditional integer quantization techniques. It details the inefficiencies in current floating-point implementations (large word size, general fixed-point and floating-point machinery, IEEE 754 specific machinery) and proposes solutions such as reducing word size through tapered floating point (inspired by posits) and avoiding complex multipliers/dividers by using fraction maps. The research paper 'Rethinking floating point for deep learning' and open-source hardware designs/software are referenced.

Nov 2018Open-sourcing FBGEMM for state-of-the-art server-side inferenceThis post introduces and open-sources FBGEMM, a high-performance kernel library specifically optimized for server-side inference of deep learning models. It focuses on efficient low-precision calculations (e.g., INT8, FP16) and matrix-matrix multiplications (GEMM), which are critical for fully connected operators in deep learning models. FBGEMM achieves over 2x performance gains compared to the existing production baseline by optimizing for contemporary CPU architectures, exploiting cache locality, and fusing compute with bandwidth-bound operations like packing. It also supports accuracy-loss-minimizing techniques like row-wise quantization and outlier-aware quantization. The library is deployable with Caffe2 and will soon be callable by PyTorch 1.0.

Nov 2018Open-sourcing foundational tools for AI performanceOpen-sourced QNNPACK and FBGEMM, high-performance kernel libraries for optimizing AI model inference. QNNPACK enables efficient computer vision model performance on mobile devices, while FBGEMM speeds up server-side models for language translation, recommendation systems, and text understanding. These libraries facilitate the use of low-precision integer arithmetic for inference, improving performance without significant accuracy loss.

Nov 2018Zero-shot learning: Using text to more accurately identify imagesIntroduced a novel generative adversarial zero-shot learning (GAZSL) model that uses GANs to identify objects described in text. This model extracts visual features from text, creates synthetic visualizations, and uses them to classify objects it hasn't been trained on. Tested against seven other ZSL algorithms, GAZSL achieved 4-7% higher accuracy across four benchmarks, demonstrating a more accurate and generalizable approach to computer vision.

Oct 2018QNNPACK: Open source library for optimized mobile deep learningThis post introduces QNNPACK, an open-source library of high-performance kernels optimized for mobile deep learning inference. QNNPACK focuses on quantized neural networks, providing efficient implementations of convolutional and fully connected operations on 8-bit integer tensors. It achieves significant performance gains (up to 2x) on mobile devices by optimizing for low-precision computations and leveraging the PDOT microkernel with fused requantization, enabling it to fit entire panels of input matrices into L1 cache and eliminate unnecessary memory transformations. QNNPACK supports common neural network primitives like grouped and depthwise convolutions, which were previously underutilized on mobile due to performance limitations.

Oct 2018Facebook, NYU expand available languages for natural language understanding systemsIntroduced the XNLI dataset, expanding NLU capabilities to 14 additional languages (including Swahili and Urdu) by building on the MultiNLI corpus. Provided 112,500 annotated sentence pairs and baseline models to evaluate cross-lingual sentence representations and facilitate research in non-English-centric NLU.

Oct 2018Significantly faster generation and training for AI-based audio systemsIntroduced SING (Symbol-to-Instrument Neural Generator), a neural audio synthesizer that generates musical notes with significantly faster training and audio generation times compared to existing AI networks. SING produces audio in larger batches (up to 1,024 samples at once) and achieves generation speeds 2,500x faster and training speeds 32x faster than comparable systems like WaveNet, while maintaining high realism.

Oct 2018Improving AI language understanding by combining multiple word representationsIntroduced dynamic meta-embeddings, a novel approach to natural language processing that dynamically selects the most useful word embeddings for a given task by training a neural network on multiple embeddings. This approach outperforms traditional models using a single type of word embedding and provides insights into how NLP systems prioritize embeddings. The code for dynamic meta-embeddings has been open-sourced.

Oct 2018Under the hood: Facebook Marketplace powered by artificial intelligenceThis post details the application of AI and ML to Facebook Marketplace, focusing on building a multimodal product index using Lumos (image understanding) and DeepText (text understanding). It describes concatenating image encoder outputs (ResNet) and text embeddings into a complete product model, and incorporating a buyer model based on demographic and search data. Cosine similarity is used for ranking. FAISS is employed for efficient similarity search to reduce computing load. The post also discusses distributed training of neural networks for millions of products, improving QPS by tuning classification threads, and optimizing memory purging with `MALLOC_CONF`. For sellers, AI is used for autosuggesting categories and pricing, and enhancing image lighting. The 'M' assistant in Messenger uses AI for intent recognition, entity extraction (Duckling), and suggestion ranking, including translation via NMT. Future work includes visual search from real-world photos and recommending complementary products.

Oct 2018Facebook accelerates AI development with new partners and production capabilities for PyTorch 1.0This post announces PyTorch 1.0, which bridges the gap between AI research and production deployment. Key technical contributions include a hybrid front end for tracing and scripting models, a revamped torch.distributed library for faster training, and an eager mode C++ interface. It highlights expanded ecosystem support from cloud providers and hardware vendors, enabling easier development, training, and deployment of deep learning models across diverse hardware. The post also mentions educational partnerships to promote PyTorch adoption.

Oct 2018Training AI agents to solve unfamiliar tasksIntroduced composable planning as a new approach to building AI agents capable of solving unfamiliar tasks. This method trains agents on a set of simple, related tasks within a given environment to enable them to perform longer, more complex tasks by learning an environment model based on assigned attributes and then using this model to plan novel tasks. This research aims to create more versatile AI agents beyond single-purpose systems.

Sep 2018Data distillation makes omni-supervised learning possibleThis post introduces and explores 'data distillation' as a method for omni-supervised learning. It details a four-step process: training on supervised data, applying to unsupervised data, generating labels for unsupervised data, and retraining with both. The experiments demonstrate that omni-supervised learning can surpass fully supervised learning, suggesting that combining curated and unlabeled data is a viable strategy for accelerating AI system creation.

Sep 2018Efficient tuning of online systems using Bayesian optimizationThis post introduces the application of Bayesian optimization, an AI technique, to efficiently tune the numerous parameters within Meta's large backend systems, particularly machine learning models. It details how this method adaptively designs A/B tests based on prior results, enabling joint tuning of more parameters with fewer experiments compared to traditional methods like grid search or manual tuning, and has been successfully applied to dozens of parameter tuning experiments.

Sep 2018Glow: A community-driven approach to AI infrastructureThis post announces the open-sourcing of Glow, a community-driven compiler framework designed to enable hardware acceleration for Machine Learning (ML) workloads. Glow accepts computation graphs from ML frameworks like PyTorch and generates optimized code for ML accelerators. It features hardware-independent optimizations, a configurable memory allocator, a linear algebra optimizer, and a CPU-based reference implementation for accuracy testing. The goal is to reduce time-to-market for hardware partners and extend hardware acceleration beyond data centers to mobile devices.

Sep 2018Expanding automatic machine translation to more languagesThis post details the expansion of Meta's automatic machine translation services to include 24 new languages, addressing the challenge of low-resource languages. It outlines three key strategies: 1) Increasing labeled in-domain data by manually labeling millions of words in 25 languages, showing an average BLEU score increase of +7.2. 2) Exploring semi-supervised NMT techniques like back-translation (average +2.5 BLEU) and copy-target (average +2.7 BLEU), and mining translations from multilingual webpages (average +0.4 BLEU). It also discusses the importance of in-domain data up-sampling to mitigate bias and improve fluency. 3) Utilizing multilingual modeling by combining related language directions, which beat bilingual baselines by +4.6 BLEU on average. The post also mentions the use of PyTorch Translate, ONNX, and Caffe2 environments for training and production.

Sep 2018Rosetta: Understanding text in images and videos with machine learningThis post details the development and deployment of Rosetta, a large-scale machine learning system for extracting and understanding text from images and videos. It describes the two-step process of text extraction: detection using a modified Faster R-CNN with a ShuffleNet-based architecture and region proposal network (RPN) with wider anchors, and recognition using a CNN based on ResNet18 with CTC loss. The post highlights the use of curriculum learning and learning rate scheduling for training the sequence prediction model, and a trick for handling right-to-left languages. It also discusses the use of a synthetic data generation engine (inspired by SynthText) for training, which is Unicode compatible and supports special rules for languages like Arabic. The inference process runs on CPU machines with Caffe2 for real-time latency.

Sep 2018Scaling neural machine translation to bigger data sets with faster training and inferenceThis post details significant optimizations to neural machine translation (NMT) training speed, reducing training time from weeks to 32 minutes. Key contributions include: switching to 16-bit precision with overflow monitoring, implementing cumulative gradient updates to reduce GPU communication overhead and increase effective batch size, overlapping backward passes with gradient synchronization for distributed training, and leveraging larger monolingual datasets with back-translation to improve accuracy. Inference speed was also improved through caching and batching by word count.

Aug 2018Unsupervised machine translation: A novel approach to provide fast, accurate translations for more languagesIntroduced a novel unsupervised machine translation approach that significantly improves accuracy (over 10 BLEU points) for low-resource languages by combining word-by-word initialization (derived from aligned word embeddings), language modeling on monolingual data, and iterative back-translation. This method achieves performance comparable to supervised models trained with substantial parallel data, opening possibilities for broader AI applications beyond machine translation.

Aug 2018Facebook and NYU School of Medicine launch research collaboration to improve MRIThis post details a research collaboration between Facebook AI Research (FAIR) and NYU School of Medicine to investigate the use of AI for accelerating MRI scans. The project, named fastMRI, aims to train neural networks to reconstruct high-quality MRI images from less collected data, potentially reducing scan times by up to 10 times. The collaboration leverages FAIR's deep learning and large-scale computing expertise with NYU's imaging science expertise. A dataset of 10,000 clinical cases (approx. 3 million images) from NYU, stripped of patient information, will be used. The project plans to open-source AI models, baselines, and evaluation metrics, while NYU will open-source the image dataset.

Jul 2018Facebook researchers win Test of Time awardThis post highlights the 2018 Test of Time award for the 2008 paper 'A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning'. The paper introduced a deep neural network approach to NLP, integrating tasks like part-of-speech tagging, chunking, parsing, and semantic-role labeling into a single jointly trained system. Key contributions included semantically meaningful word vectors, multitask learning, convolutional neural nets in NLP, and an attention mechanism. The post emphasizes how this research laid the groundwork for current dominant neural network approaches in NLP, influencing tools like Fairseq and Elmo.

Jul 2018Talk the Walk: Teaching AI systems to navigate New York through languageIntroduced the 'Talk the Walk' research task for embodied AI, using 360-degree images of New York City for navigation. Developed a novel attention mechanism called MASC (Masked Attention for Spatial Convolution) to improve the guide bot's ability to focus on the map and predict the tourist bot's location, achieving over twice the accuracy on the test set. Explored emergent communication protocols (continuous vectors and synthetic language) and compared their performance against natural language, finding synthetic communication to be more efficient but less human-understandable. Released baselines and the dataset for 'Talk the Walk' to the research community.

May 2018Embodied Question Answering: A goal-driven approach to autonomous agentsIntroduced Embodied Question Answering (EmbodiedQA), a new AI task requiring agents to combine perception, navigation, and communication to answer questions in virtual environments. Developed House3D, a collection of 45,000 simulated indoor environments for training and testing these agents. Proposed a modular navigation approach splitting tasks between a planner and controller. Explored learning capabilities including active perception, commonsense reasoning, language grounding, and credit assignment using reinforcement learning, imitation learning, and reward shaping. Released EmbodiedQA data and House3D as open source.

May 2018AI at F8 2018: Open frameworks and responsible developmentThis post details the evolution of Meta's AI development by announcing PyTorch 1.0 for seamless research-to-production transition and the expansion of ONNX to support more frameworks for model interoperability. It also highlights the release of specific AI models and libraries such as ResNext3D for video understanding, Translate for machine translation, and MUSE for multilingual embeddings. Research advancements include a new image recognition model trained on 3.5 billion images, DensePose for 3D body mapping, EmbodiedQA for autonomous agents, and open-source Go and StarCraft bots (ELF OpenGo and StarCraft bot tools) for reinforcement learning research. The post also emphasizes responsible AI development through efforts to eliminate algorithmic bias and ensure ethical deployment.

May 2018Announcing PyTorch 1.0 for both research and productionAnnounces PyTorch 1.0, which unifies PyTorch's research flexibility with Caffe2's production capabilities and ONNX interoperability. This provides a seamless path from research prototyping to production deployment for AI projects. Key features include a hybrid front-end for imperative and declarative execution, native ONNX export, and the open-sourcing of related tools like Translate and ELF.

May 2018Advancing state-of-the-art image recognition with deep learning on hashtagsThis post details a novel approach to training image recognition models by leveraging large-scale public image datasets with hashtags as weakly supervised labels, rather than relying solely on labor-intensive manual annotation. It describes the challenges of noisy and vague hashtags and the development of new methods to handle multiple labels per image, sort synonyms, and balance frequent/rare hashtags. The work achieved record-high accuracy on ImageNet (85.4%) by training on 1 billion images with 1,500 matched hashtags, and demonstrated significant performance boosts (up to 22.5% relative improvement) on COCO object detection. The post also highlights the use of distributed training across 336 GPUs to shorten training time from over a year to a few weeks and the importance of selecting relevant hashtags for specific tasks. Future directions include using hashtags for video understanding, image ranking, and recognizing specific subcategories within images.

May 2018ONNX expansion speeds AI developmentThis post details the expansion of the ONNX format to support additional AI tools like Baidu's PaddlePaddle, Qualcomm SNPE, and Apple's Core ML. It highlights how ONNX facilitates interoperability between deep learning frameworks and hardware runtimes, accelerating the research-to-production pipeline for AI engineers. New integrations with NVIDIA TensorRT 4 and community projects are also mentioned, along with future plans for NLP support and a common framework interface.

Mar 2018The next step in Facebook’s AI hardware infrastructureThis post announces Big Basin v2, the next iteration of Meta's AI hardware infrastructure. Big Basin v2 builds upon the modular design of its predecessor, incorporating the latest NVIDIA Tesla V100 GPU accelerators, an upgraded head-node (Tioga Pass) for increased CPU performance, doubled PCIe bandwidth between CPUs and GPUs, and an upgraded OCP network card for enhanced distributed training bandwidth. These upgrades resulted in a 66% increase in single-GPU performance and near-linear performance improvement in large-scale distributed GPU training. The post also details how Big Basin v2 integrates with the FBLearner platform (Feature Store, Flow, Predictor) and other OCP-contributed hardware like Bryce Canyon and Twin Lakes.

Feb 2018Under the hood: Suicide prevention tools powered by AIThis post details the application of AI and machine learning for suicide prevention on Facebook. It describes the evolution from keyword-based detection to a machine learning approach using n-gram-based linear regression and later DeepText-based neural networks. The system incorporates contextual signals from comments and time of day, and uses FBLearner for training classifiers. It also discusses multilingual classification strategies and emphasizes the continued role of human review.

Jan 2018Under the hood: Multilingual embeddingsIntroduced a technique for training multilingual word embeddings by projecting language-specific embeddings into a common vector space (English) using a learned orthogonal matrix. This approach enables text classification models to generalize to unseen languages with accuracy close to language-specific models and significantly reduces latency compared to translation-based methods. The method was integrated into the DeepText framework and demonstrated improvements in shipping AI-powered features to new languages faster and with better accuracy.

Dec 2017ONNX V1 releasedThis post announces the production-readiness of ONNX (Open Neural Network Exchange format) V1, a collaborative effort with AWS and Microsoft. It highlights the goal of creating an open, flexible standard for deep learning frameworks and tools to interoperate, enabling developers to combine tools and easily transfer models. The post lists initial framework and hardware support, including Caffe2, Cognitive Toolkit, MXNet, PyTorch, TensorRT, AMD, ARM, IBM, Intel, Huawei, NVIDIA, and Qualcomm. Future versions are planned to expand beyond vision to areas like language modeling.

Sep 2017SLAM: Bringing art to life through technologyThis post details the development and deployment of Simultaneous Localization and Mapping (SLAM) technology for mobile devices, enabling immersive AR experiences. It highlights the engineering challenges in optimizing SLAM algorithms for diverse mobile devices, reducing binary size, and creating believable AR experiences through the WorldTracker API. The post also showcases the first AR-driven art project with artist Heather Day, demonstrating the potential of AR to blend art and technology.

Aug 2017Transitioning entirely to neural machine translationThis post details the transition from phrase-based statistical machine translation to neural machine translation (NMT) systems across all of Facebook's backend translation systems. It introduces the use of sequence-to-sequence LSTMs with attention for improved context handling, techniques for handling unknown words by passing source words through and looking them up in a bilingual lexicon, and vocabulary reduction to speed up computation. The post also discusses the challenges and solutions for tuning model parameters at scale using the FBLearner Flow platform. Crucially, it highlights the implementation of these NMT models using Caffe2, detailing memory optimizations for training and inference, and the use of beam search for decoding. Finally, it mentions the integration of research from CNNs for machine translation, achieving significant BLEU score improvements.

Jul 2017Using AI for new visual storytelling techniques in VRThis post details the application of AI style transfer techniques to generate special effects for a VR film ('Alteration'). It describes the challenges of applying style transfer to high-definition, 360-degree, stereoscopic VR content, including memory and time constraints, and the subjective nature of optimizing against aesthetic opinions. The post outlines the process of retraining neural networks with larger image resolutions (768x768) for VR frames, the generation of high-resolution test images, and the application of models to video frames using CPUs due to RAM constraints. It also covers the technical considerations for stereoscopy, using disparity maps to ensure consistency between eyes. The work demonstrates AI's maturity as a creative tool for filmmakers.

Jul 2017Introducing ELF: An extensive, lightweight and flexible platform for game researchIntroduced ELF (Extensive, Lightweight, and Flexible) platform for game research to democratize AI research. ELF allows researchers to test algorithms in various game environments (board games, Atari, custom RTS games) with reduced computational needs (e.g., laptop with GPU, 6 CPUs and 1 GPU for RTS in one day). It handles concurrency, has a Python UI for batching game states, and supports pairing flexibility between game instances and actor models. Included Mini-RTS, a fast RTS engine for testing ELF. ELF demonstrated faster simulation and 30% faster training speed than OpenAI Gym in Atari Games.

Jun 2017Deal or no deal? Training AI bots to negotiateIntroduced dialog agents with the capability to negotiate, utilizing multi-issue bargaining tasks. Key technical innovation is 'dialog rollouts,' a technique where an agent simulates future conversations to choose utterances with maximum expected future reward. Agents were trained end-to-end on human negotiation data and fine-tuned using reinforcement learning, achieving performance comparable to human negotiators and demonstrating novel negotiation tactics and sentence generation.

Jun 2017Accelerating machine learning for computer visionThis post details how Facebook researchers reduced the time to train the ImageNet-1k dataset from multiple days to one hour with leading classification accuracy. This was achieved through creative infrastructure design, utilizing Caffe2 and the Gloo library for collective communication, and employing Big Basin, Facebook's next-generation GPU server, which was contributed to the Open Compute Project. This work enables machine learning researchers to experiment more efficiently, driving the evolution of technologies like face filters, 360 video, and augmented reality.

May 2017ParlAI: A new software platform for dialog researchIntroduced ParlAI, an open-source platform for dialog research, unifying tasks and training algorithms in a single repository. Integrated Mechanical Turk for data collection, training, and evaluation. Categorized dialog tasks into question answering, sentence completion, goal-oriented dialog, chit-chat dialog, and visual dialog. Defined core concepts: worlds, agents, and teachers, using a unified observation/action object format. Included initial agents like IR baselines, an end-to-end memory network (Lua Torch), and an attentive LSTM model DrQA (PyTorch).

May 2017A novel approach to neural machine translationIntroduced a novel convolutional neural network (CNN) approach for neural machine translation, achieving state-of-the-art accuracy and nine times the speed of recurrent neural systems. The CNN architecture leverages hierarchical processing and parallel computation on GPUs, overcoming the sequential limitations of RNNs. Key architectural components include multi-hop attention for improved translation accuracy and gating mechanisms for controlling information flow. The FAIR sequence modeling toolkit (fairseq) and trained systems were open-sourced.

May 2017Expanded fastText library now fits on smaller-memory devicesThis post details the expansion of the fastText library to support smaller-memory devices by reducing model memory usage from gigabytes to hundreds of kilobytes. This was achieved through collaboration with FAISS and the "FastText.zip: Compressing Text Classification Models" paper, enabling deployment on smartphones and Raspberry Pi. The post also highlights fastText's efficiency, speed (1,000 to 10,000 times faster than neural networks), and its use of hierarchical softmax and bag-of-words/n-gram models for efficient text classification and word representation learning. Two self-paced tutorials are provided for supervised text classification and word representation learning.

Mar 2017Faiss: A library for efficient similarity searchThis post introduces Faiss, a new library for efficient similarity search on billion-scale datasets. It details the challenges of similarity search with high-dimensional vectors generated by AI models, compares Faiss to existing software packages, and outlines its advantages in speed, memory usage, and GPU implementation. The post also discusses evaluation metrics (speed, memory, accuracy) and presents performance benchmarks on billion-scale datasets using both CPU and GPU implementations, including the construction of k-nearest-neighbor graphs.

Mar 2017Evaluating boosted decision trees for billions of usersThis post details optimizations for gradient-boosted decision trees (GBDTs) to improve their evaluation efficiency for real-time ranking systems serving billions of users. It compares naive flat tree implementations with compiled C++ versions, introducing techniques like LIKELY/UNLIKELY annotations for branch prediction, range-based evaluation to improve CPU cache utilization, and optimizations for common and categorical features. Computational results show up to a 5x improvement over the flat tree implementation, enabling the ranking of more examples or larger models with the same resources. These techniques have been applied to notifications filtering, feed ranking, and suggestions at Facebook.

Mar 2017Introducing Big Basin: Our next-generation AI hardwareIntroduced Big Basin, a next-generation GPU server for AI hardware, as the successor to Big Sur. Big Basin offers a 30% increase in trainable model size due to greater arithmetic throughput and a memory increase from 12 GB to 16 GB. It features a modular, disaggregated design separating CPU compute from GPUs, enabling independent scaling and improved serviceability. The system utilizes NVIDIA Tesla P100 GPU accelerators connected via NVLink in a hybrid cube mesh, offering improved deep learning training performance and higher single-precision (10.6 teraflops) and half-precision floating-point arithmetic. Open-sourced the design through the Open Compute Project.

Feb 2017Building scalable systems to understand contentThis post details the development and scaling of FBLearner Flow, a general-purpose platform for machine learning pipelines, and Lumos, a computer vision platform built on top of FBLearner Flow. It highlights the transition of FBLearner Flow from a research project to a production-scale system supporting 1.2 million AI experiments per month. Lumos is presented as the engine for image and video understanding, enabling engineers to train and deploy models without deep learning expertise. Specific applications discussed include automatic alt text (AAT) for the visually impaired, with an update to include actions, and visual search capabilities powered by object recognition and image embeddings. The post emphasizes the iterative nature of model improvement through data labeling and the use of semantic clustering for data generation.

Dec 2016Artificial intelligence, revealedThis post introduces the fundamental concepts of Artificial Intelligence (AI), explaining its pervasive nature in everyday life and its potential societal impact. It demystifies AI by describing it as a rigorous science focused on designing intelligent systems using algorithmic techniques, often inspired by the brain. The post details artificial neural networks, their learning process through modifying connections, and their ability to recognize patterns. It specifically highlights deep learning and convolutional neural networks (ConvNets) for their hierarchical pattern recognition capabilities in images, speech, and text. The post also categorizes AI learning approaches into reinforcement learning, supervised learning, and unsupervised learning, discussing their characteristics and limitations. Finally, it touches upon the future job market related to AI and provides guidance for students and professionals looking to enter the field.

Nov 2016Accelerating innovation and powering new experiences with AIThis post details the evolution of Meta's AI efforts, highlighting the transition from research to production through platforms like FBLearner Flow, AutoML, and Lumos. It introduces Caffe2Go, a mobile deep learning platform enabling real-time style transfer on devices by condensing AI models by 100x. It also discusses AI's application in VR for stabilization and UI, and in connectivity projects for deployment planning using computer vision and 3D city analysis. The post outlines future research challenges in contextual understanding and reasoning for AI systems.

Nov 2016Delivering real-time AI in the palm of your handThis post details the development and deployment of Caffe2Go, a lightweight, modular deep learning framework designed for real-time AI inference on mobile devices (iOS and Android). It describes optimizations for model size and quality for style transfer, including reducing convolution layers, adjusting layer width, and spatial resolution. The framework's cross-platform compatibility (server CPU, GPU, iOS, Android) and developer-friendly deployment process are highlighted. Performance improvements of 100x in model size and inference times of <50ms on mobile phones are achieved.

Oct 2016Building an efficient neural language model over a billion wordsThis post introduces adaptive softmax, a novel softmax approximation technique tailored for GPUs to efficiently train neural network based language models over very large vocabularies. It also details the open-sourcing of torch-rnnlib, a library that allows researchers to design new recurrent models and test prototypes on GPUs with minimal effort, including seamless access to fast baselines through torch.cudnn bindings. The post demonstrates achieving a perplexity of 45 on the 1-billion word dataset on a single GPU in a couple of days using these tools.

Aug 2016Segmenting and refining images with SharpMaskIntroduced DeepMask, a segmentation framework that treats segmentation as a series of binary classification problems for image patches and pixels. Developed SharpMask, a refinement module that uses earlier network layer features to improve the fidelity of object masks generated by DeepMask. Proposed MultiPathNet, a specialized convolutional network architecture designed to exploit information at multiple image scales and surrounding context for improved object classification accuracy, building upon the RCNN paradigm.

Aug 2016FAIR open-sources fastTextThis post introduces fastText, an open-source library for efficient text classification and learning word vector representations. It details the technical approach, including the use of hierarchical softmax for speed, bag-of-n-grams, subword information, and sharing hidden representations across classes. It also highlights its efficiency on large datasets and its ability to learn word representations that outperform word2vec, especially for morphologically rich languages.

Jun 2016Lighting the way to deep machine learningIntroduced and open-sourced Torchnet, a software toolkit for the Torch community to foster rapid and collaborative development of deep learning experiments. Torchnet provides boilerplate code, key abstractions, and reference implementations that can be snapped together or taken apart and reused, speeding up development. It encourages a modular programming approach, reducing bugs and enabling asynchronous, parallel data loading and efficient multi-GPU computations. Torchnet provides five main abstractions: Datasets, Dataset Iterators, Engines, Meters (for performance measurements), and Logs. It also includes subpackages for computer vision, natural language processing, and speech processing, with specific wrappers for datasets like Imagenet, COCO, TIMIT, LibriSpeech, and text datasets.

Jun 2016A path to unsupervised learning through adversarial networksThis post details advancements in stabilizing Generative Adversarial Networks (GANs), specifically introducing Laplacian Adversarial Networks (LAPGAN) and Deep Convolutional Generative Adversarial Networks (DCGAN) for image generation, and Adversarial Gradient Difference Loss Predictors (AGDL) for video generation. It highlights how these techniques enable unsupervised learning by allowing machines to build internal models of the world through prediction, bypassing the need for human-designed cost functions. The post also discusses visualizing learned filters in DCGANs to understand their internal workings and demonstrates their ability to learn object attributes and semantic relationships without supervision.

Jun 2016Introducing DeepText: Facebook’s text understanding engineThis post introduces DeepText, a deep learning-based text understanding engine for Facebook. It details the use of convolutional and recurrent neural networks, word embeddings, and semi-supervised learning to achieve near-human accuracy in understanding text across multiple languages. The post highlights the challenges of traditional NLP for text understanding at scale and in diverse languages, and how deep learning addresses these. It also describes the application of DeepText in Messenger for intent detection and for identifying items for sale, and outlines future research directions including joint text-visual understanding and new neural network architectures.

May 2016Artificial intelligence, revealedThis post introduces fundamental AI concepts including Machine Learning, Gradient Descent, Deep Learning, Back Propagation, Convolutional Neural Networks, and their applications at Meta for tasks like text translation and image description for the visually impaired. It emphasizes the mathematical underpinnings and layered processing involved in AI.

May 2016Introducing FBLearner Flow: Facebook’s AI backboneIntroduced FBLearner Flow, a new platform designed to democratize machine learning model training and experimentation for all Facebook engineers. FBLearner Flow enables reusable algorithms, parallelized training pipelines, automated experimentation, and easy search and sharing of past experiments. Key components include workflows, operators, and channels. The platform features an authorship and execution environment for custom distributed workflows and an experimentation management UI for launching experiments and viewing results. It has enabled over a million models to be trained and supports millions of predictions per second.

Apr 2016Powering Facebook experiences with AIThis post details the application of AI and machine learning to enhance core Facebook experiences, including automatic translation for non-English speakers, photo image search based on content, image segmentation for object recognition and accessibility features, and real-time video classification. It highlights the use of a large GPU cluster (40 PFLOPS) for training models with billions of parameters on trillions of examples, leading to a 50x increase in AI experiments per day and faster research-to-production cycles. The post emphasizes the development of custom models tailored to Facebook's unique platform language and user-generated content.

Mar 2016Engineering for nostalgia: Building a personalized “On This Day” experienceThis post details the engineering behind Meta's 'On This Day' feature, focusing on personalization and content understanding. It describes the use of UX research to inform filtering and ranking algorithms. Specifically, it highlights the application of automatic filters (e.g., for exes, blocked users, dismissed memories) and user-defined preferences (blocklisting people or dates). The core technical contribution lies in the machine-learning ranking model, which uses personalization signals (past interactions, demographics, memory attributes) and content understanding signals from a computer vision platform (object, scene, action, place recognition) to surface meaningful memories. The computer vision platform is built on a deep convolutional neural network trained on millions of examples.

Dec 2015Facebook to open-source AI hardware designThis post introduces 'Big Sur', Meta's next-generation GPU-based system for training neural networks at scale. Big Sur is an Open Rack-compatible 8-GPU server designed for AI computing, incorporating high-performance GPUs and flexible PCI-e topologies. It offers twice the performance of previous generations and allows for further scaling through distributed training. The hardware is optimized for thermal and power efficiency, enabling operation in free-air cooled data centers. It also features improved serviceability with toolless component replacement. Meta plans to open-source the Big Sur design and contribute it to the Open Compute Project.

Nov 2015Teaching machines to see and understand: Advances in AI researchThis post details advancements in AI research at Meta, focusing on object detection and memory networks. The object detection system achieves 30% faster segmentation and uses 10x less training data. Memory Networks (MemNets) are enhanced to handle datasets exceeding 100K questions, an order of magnitude larger than previous benchmarks. A Visual Q&A (VQA) system combining MemNets with image recognition is demonstrated. Further research areas include unsupervised and predictive learning, with a system predicting block stacking outcomes with 90% accuracy, and an AI Go player that combines search-based approaches with pattern matching, achieving parity with strong human players. The AI assistant 'M' is introduced as a testbed for integrating these capabilities to handle user requests autonomously.

Jun 2015Oculus scientist wins computer vision awardThis post highlights a significant achievement in computer vision research with the award-winning paper 'DynamicFusion: Reconstruction and Tracking of Non-rigid Scenes in Real-Time'. This work represents a breakthrough in real-time 3D scanning of moving scenes, directly contributing to advancements in virtual and augmented reality by enabling more realistic integration of the real world into these digital environments.

Jun 2015Recommending items to more than a billion peopleThis post details the challenges and innovations in scaling collaborative filtering (CF) for recommendation systems at Facebook's massive scale. It introduces the problem of handling billions of ratings and users, discusses the limitations of standard distributed approaches for matrix factorization (SGD and ALS), and presents a novel 'rotational hybrid approach' that significantly reduces network traffic and overcomes skewed item degree distributions by extending the Apache Giraph framework with worker-to-worker messaging. It also discusses the blend of SGD and ALS for improved performance and accuracy.

Jan 2015FAIR open sources deep-learning modules for TorchThis post announces the open-sourcing of optimized deep-learning modules for Torch, developed by FAIR. These modules include GPU-optimized layers for large convolutional nets (ConvNets) and networks with sparse activations, featuring a fast FFT-based convolutional layer using custom CUDA kernels built around NVIDIA's cuFFT library. It also introduces containers for multi-GPU parallelization (data-parallel and model-parallel), an optimized Lookup Table, Hierarchical SoftMax, Cross-map pooling, a GPU implementation of 1-bit SGD, and a faster Temporal Convolution layer. The FFT-based convolutional layer achieves speedups of up to 23.5x compared to CuDNN for larger kernel sizes, and the multi-GPU parallelization aims to simplify training over multiple GPU cards.

Dec 2011Meet a Facebook Fellow: Marek OlszewskiIntroduced Kendo, a software-only prototype system providing a deterministic multithreading programming model. Kendo enforces deterministic interleaving of synchronization operations via a novel distributed scheduling algorithm that uses performance counters to construct deterministic logical clocks for load-balanced scheduling. It runs on commodity hardware with modest performance cost, simplifying debugging and testing of parallel applications by enabling repeatable results and systematic bug detection.

What this trail covers