Cerebras Software Platform R1.2 is Out! - Cerebras

CSoft 1.2 introduces "Weight Streaming" for extreme-scale AI models, enabling independent scaling of model size and training performance, and supporting models up to 120 trillion parameters. It adds support for training GPT3-XL (1.3 billion parameters) on a single CS-2 system. The release also expands PyTorch support to version 1.10.2 using a lazy tensor backend with XLA, allowing for efficient mapping of full model graphs onto the WSE-2 and enabling training, evaluation, and fine-tuning of NLP models like BERT, GPT2, Transformers, and T5 with significantly improved performance.