Sparsity Made Easy – Introducing the Cerebras PyTorch Sparsity Library - Cerebras

Introduces a new Cerebras PyTorch Sparsity Library that simplifies the use of unstructured sparsity for training large language models on Cerebras hardware. The library provides a modular API for implementing static and dynamic sparsity algorithms, including GMP, SET, and RigL, and demonstrates ease of integration with existing PyTorch models and the Cerebras Model Zoo. Benchmarks show improved model quality with dynamic sparsity over static methods on a Llama2-like model. The library's extensibility is highlighted by enabling the GraNet algorithm.