Internationalization and Language Modeling

5/9/2022

This post details the development of a custom mobile localization infrastructure at Meta, featuring downloadable language packs and the FBT (Facebook Translation) framework. It addresses challenges with native Android/iOS localization by supporting gendered text and reducing app size. The post explains the end-to-end flow, construction of language packs, FBT string creation with extraction and structure, and the client-side adoption of language packs, including improvements for poor network conditions like prefetching and fallback mechanisms.

9/27/2019

This post details the technical implementation of font converters in Facebook and Messenger to support Myanmar's transition from Zawgyi to Unicode. It explains the challenges of content encoding detection (differentiating Zawgyi from Unicode using ML models trained on Facebook content), device encoding detection (using string width on-device), and the conversion process itself. It also highlights the use of Google's myanmar-tools library and the need to train models on Facebook-specific content due to its informal nature.

9/11/2018

This post details the expansion of Facebook's automatic machine translation (NMT) services to include 24 new languages. It describes the technical challenges of low-resource language translation, including lack of training data and the need for fast training. The post outlines three key strategies employed: 1) Increasing labeled in-domain data by manually labeling millions of words in 25 languages, showing an average BLEU score increase of +7.2. 2) Exploring semisupervised NMT techniques like back-translation (average +2.5 BLEU), copy-target (average +2.7 BLEU), and mining translations from multilingual webpages (average +0.4 BLEU), along with in-domain data up-sampling for improved fluency and relevance. 3) Utilizing multilingual modeling by combining related language directions, which improved BLEU scores by an average of +4.6. The post also mentions the use of PyTorch Translate, ONNX, and Caffe2 environments for training and production.

9/7/2018

This post details significant advancements in scaling neural machine translation (NMT) by drastically reducing training and inference times. Key contributions include achieving a 45x speedup in training through optimizations like 16-bit precision, cumulative updates, and overlapping communication with backward passes. It also highlights the ability to train more accurate models using larger datasets, including monolingual data via back-translation, and improvements in translation speed. The post emphasizes the open-sourcing of the fairseq library to enable faster NMT training for the wider research community.

1/24/2018

Introduced multilingual embeddings for NLP tasks, enabling text classification across languages without explicit translation. This involves training language-specific embeddings using fastText, projecting them into a common space using an orthogonal matrix derived from parallel data, and integrating them into the DeepText framework. This approach significantly improves cross-lingual accuracy and reduces latency compared to translation-based methods.

8/3/2017

This post details the significant transition from phrase-based statistical machine translation to neural machine translation (NMT) for all of Facebook's backend translation systems. It highlights the use of sequence-to-sequence LSTMs with attention for improved context handling and accuracy, techniques for managing unknown words and vocabulary reduction, and the scaling of these models using Caffe2. The post also introduces the application of convolutional neural networks (CNNs) for translation and the ongoing exploration of multilingual models, marking a major leap in translation quality and efficiency.

7/1/2016

Introduced a new multilingual composer for Pages and began testing for individual users, allowing a single post to be written in multiple languages. This involved developing new composer components using React and Flux, a storage implementation using 'author translations' (separate TAO objects for each language) instead of concatenation, and a server-side viewing experience that determines the most relevant language version for each viewer based on language preferences, locale, and common posting language. Machine translation is used to pre-fill additional language versions.

2/4/2012

This post details Ben Gertzfield's work on enabling non-English characters in fan page URLs, which improved search ranking for international content. It also discusses the virtuous cycle of user-driven translation and the reuse of code between products, specifically mentioning the application of MQTT from space probes to Facebook Messenger for efficient real-time messaging.

9/8/2009

This post details the technical implementation of Facebook's community-driven translation application, focusing on handling "Named Entities" within phrases. It introduces "Glossary" for consistent vocabulary, "Dynamic Explosion" to manage language-specific variations based on token values (e.g., gender, number), and "Linguistic Rules" for orthographic and phonological adjustments. The post also discusses the development of over 250 million individual language models to capture user dialects and their applications in spam detection and speech recognition, and mentions dialectometrical methods on the social graph.