Blogs›Meta›Agentic Development and Just-in-Time Testing
Meta is exploring agentic development and introducing Just-in-Time Tests (JiTTests) as a novel approach to software testing. JiTTests leverage LLMs to automatically generate tests on-the-fly for specific code changes, aiming to catch bugs just before they enter production. This approach contrasts with traditional testing methods that rely on manually authored, maintained, and executed test suites. The JiTTest process involves inferring code change intent, creating mutants (code with deliberate f. This post details the development and deployment of the Automated Compliance Hardening (ACH) tool, a system for mutation-guided, LLM-based test generation. ACH hardens platforms against regressions by generating undetected faults (mutants) in source code that are specific to a given area of concern and using those same mutants to generate tests. It automates the process of searching for privacy-related faults and preventing them from entering systems, hardening code bases to reduce the risk of privacy regressions. ACH automatically generates unit tests that target a particular kind of fault, described in plain text, and keeps verifiable assurances that its tests do catch the kind of faults described. This approach is a departure from traditional techniques that focus on code coverage, instead targeting specific faults. ACH has been applied to platforms including Facebook Feed, Instagram, Messenger, and WhatsApp.
2/11/2026
Introduces Just-in-Time Tests (JiTTests) as a new testing paradigm for agentic development. Details the JiTTest process: inferring intent, creating mutants, generating/running tests, and using assessors. Highlights benefits like reduced maintenance, tailored tests, and focus on true positives.
8/6/2025
Introduces Diff Risk Score (DRS), an AI-powered system using a fine-tuned Llama LLM to predict production incident likelihood from code changes. Details its application in enabling code unfreezes during sensitive periods, improving developer productivity and reliability. Outlines future directions including expanding to configuration change risk, automating risk mitigation with AI agents, and enhancing explainability through natural language outputs. Mentions the development of the Risk Awareness Platform (RAP).
2/5/2025
Introduced the Automated Compliance Hardening (ACH) tool, a system for mutation-guided, LLM-based test generation. ACH generates faults (mutants) in source code based on plain-text descriptions of desired bug types and then uses these mutants to automatically generate tests that are guaranteed to catch those specific faults. This approach is applied to various Meta platforms like Facebook Feed, Instagram, Messenger, and WhatsApp.
9/17/2024
This post introduces Bento, Meta's internal distribution of Jupyter Notebooks. It highlights the platform's ability to mix code, text, and multimedia for various use cases, including prototyping and ML workflows. Key features discussed are scheduled notebooks, sharing capabilities, and serverless notebook execution leveraging WebAssembly in the browser.
9/13/2018
Introduced SapFix, an AI hybrid tool for automatically generating and proposing software bug fixes. SapFix utilizes strategies such as code reversion, templated fixes, and mutation-based AST modifications. It performs autonomous validation by running existing and Sapienz-generated tests to check for compilation errors, persistent crashes, and new regressions. Approved fixes are then submitted to human reviewers for deployment.