TechPrufer
Get started
IBM
IBM logoIBM · Feature trail

AI Inferencing on IBM Power10 with MMA

1 post · Oct 2022

What this trail covers

This post introduces the capability of running AI inferencing on IBM Power10 systems leveraging the new Matrix Math Accelerator (MMA) units. It details the developer setup using Anaconda and Conda packages from the RocketCE channel, and outlines a container-based demo framework for showcasing performance gains. Six real-world scenarios are presented, demonstrating up to 3x performance improvements with MMA for various AI workloads like image detection, action recognition, and fraud detection.

All IBM trails
  1. How to run AI inferencing on IBM Power10 leveraging MMAThis post details the technical approach to running AI inferencing on IBM Power10 systems using the new Matrix Math Accelerator (MMA). It explains the developer setup process involving Anaconda and the RocketCE Conda channel, and describes a container-based demo framework architecture. Six specific demos are presented, highlighting performance improvements achieved by leveraging MMA-optimized libraries in PyTorch and ONNX runtime for image detection, action recognition, credit card fraud detection, and speech-to-text.