Open-sourcing PyText for faster NLP development

This post announces the open-sourcing of PyText, a new NLP modeling framework built on PyTorch. PyText aims to simplify the workflow for NLP development, enabling faster experimentation and easier large-scale deployment. It offers prebuilt model architectures, utilities for text processing, and access to the PyTorch ecosystem. The framework has been used internally for over a billion daily predictions and has reduced development time from weeks/months to days. It addresses the historical trade-off between experimentation-optimized and production-optimized frameworks by providing a unified approach. PyText supports conversion to ONNX and Caffe2 for production deployment, offering a modular design with features like distributed training, multitask learning, portability, prebuilt models, and contextual modeling. It also improves upon DeepText by supporting dynamic graphs and offering faster training with GPU utilization and distributed training.