TechPrufer
Get started
Cerebras
Cerebras logoCerebras · Feature trail

High-Resolution Computer Vision Training

1 post · Mar 2025

What this trail covers

The Cerebras CS-2 system enables training large deep neural networks on high-resolution images (up to 50 megapixels) without the need for downsampling or tiling, overcoming GPU memory limitations. This allows for richer contextual information to be leveraged in computer vision tasks such as medical imaging and remote sensing, leading to improved accuracy and insights. The system's on-chip SRAM and weight-streaming capabilities are key enablers. This release extends support to UNet for 2D segmentation with images up to 5K x 5K pixels, a size not supported out-of-the-box on GPUs today. Additionally, performance for GPT-style models is improved by up to 1.5x, and training with unstructured sparse weights is now available in PyTorch, enabling users to convert dense checkpoints to sparse versions for reduced FLOPs and faster iteration.

All Cerebras trails
  1. Unlocking High-Resolution Computer Vision with Wafer-Scale Technology - CerebrasThis post details Cerebras's capabilities in accelerating large image CNNs, specifically for tasks like image segmentation and classification. It highlights the advantages of the CS-2 system's Wafer-Scale Engine (WSE) and Weight Streaming execution mode for handling high-resolution images and volumes. Key contributions include demonstrating the ability to fit all compute and activations for 2D/3D segmentation/classification models on a single WSE chip, enabling training on images up to 50MP (with updates extending to 7K2 resolution), and overcoming the performance limitations of bandwidth-bound operations like Batch Normalization and Group Normalization on traditional GPU architectures. The post also elaborates on the mapping of multi-dimensional convolution tensors to the chip's spatial layout and the compiler's role in optimizing these layouts.
  2. More Pixels, More Context, More Insight! - CerebrasThis post details the Cerebras CS-2 system's ability to train large deep neural networks on high-resolution images (up to 50 megapixels) without downsampling or tiling, overcoming GPU memory limitations. It highlights the system's on-chip SRAM and weight-streaming capabilities as key enablers for medical imaging and remote sensing applications. The post also notes an update to R1.8 of the Cerebras Software Platform, which now supports image segmentation on 50 megapixel images, an increase from 25 megapixels in R1.7.
  3. Cerebras Software Platform R1.7 is Out! - CerebrasThis release (CSoft v1.7) introduces unique support for 2D segmentation with UNet, enabling training on images up to 5K x 5K pixels without tiling, a capability exceeding current GPU limitations. It also extends PyTorch support for large language models, including GPT-J and GPT-3 up to 20B parameters, with a performance improvement of up to 1.5x for GPT-style models. Furthermore, the platform now enables training large language models with sparse weights in PyTorch, providing scripts to convert dense PyTorch checkpoints to sparse versions, leveraging research on unstructured sparsity for reduced FLOPs and faster iteration.