TechPrufer
Get started
Hugging Face
Hugging Face logoHugging Face · Feature trail

ONNX Runtime Training Integration

1 post · Jun 2022 – Jan 2023

What this trail covers

Hugging Face Optimum integrates with ONNX Runtime for training and inference, offering significant speedups and optimizations for popular Hugging Face models. This includes leveraging ONNX Runtime's memory and compute optimizations, fused optimizers, mixed-precision training, and integration with DeepSpeed ZeRO Stage 1. The ORTTrainer API provides a seamless transition for users familiar with the Transformers Trainer API. This post details the conversion of Transformers models to ONNX format using `torch.onnx`, `transformers.onnx`, and the high-level Optimum library, enabling accelerated inference and further optimization steps like quantization and TensorRT conversion.

All Hugging Face trails
  1. Optimum+ONNX Runtime - Easier, Faster training for your Hugging Face modelsThis post introduces the integration of ONNX Runtime Training within Hugging Face's Optimum library. It details the performance benefits, showcasing up to 130% acceleration on selected models when using ONNX Runtime with DeepSpeed ZeRO Stage 1. The post explains the underlying optimizations of ONNX Runtime Training, such as efficient memory planning and kernel optimizations. It also introduces the `ORTTrainer` and `ORTTrainingArguments` APIs, which extend the existing `Trainer` and `TrainingArguments` to leverage ONNX Runtime for training, including fused optimizers and compatibility with DeepSpeed.
  2. Convert Transformers to ONNX with Hugging Face OptimumThis post details three methods for converting Hugging Face Transformers models to ONNX format: low-level `torch.onnx.export`, mid-level `transformers.onnx.export`, and high-level Optimum's `ORTModelForSequenceClassification.from_pretrained(..., from_transformers=True)`. It provides code examples for each method using the `distilbert-base-uncased-finetuned-sst-2-english` model and outlines potential next steps such as accelerated inference, static quantization, and TensorRT conversion.