Introducing Sparse Llama: 70% Smaller, 3x Faster, Full Accuracy - Cerebras

Introduces a collaboration with Neural Magic to achieve 70% sparsity in LLMs (specifically Llama) using a combination of one-shot pruning, sparse pretraining on SlimPajama, and fine-tuning. Highlights Cerebras CS-3's native support for arbitrary unstructured sparsity and Neural Magic's DeepSparse engine for accelerated CPU inference. Demonstrates full accuracy recovery for downstream tasks and significant parameter reduction.