RCCLX: Innovating GPU Communications on AMD Platforms

What this post added

This post introduces RCCLX, an open-sourced enhanced version of RCCL for AMD platforms, integrated with Torchcomms. It details two key innovations: Direct Data Access (DDA) for lightweight intra-node collectives, improving AllReduce latency by allowing direct memory access and reducing latency from O(N) to O(1) or a constant factor, and Low Precision Collectives (LP Collectives) which use FP8 quantization for up to 4:1 compression to reduce communication overhead and improve scalability for large message sizes. DDA shows 10-50% performance improvements on AMD MI300X GPUs for decode and 10-30% for prefill, resulting in ~10% reduction in time-to-incremental-token. LP Collectives demonstrate significant speedups for FP32 and notable improvements for BF16, with observed ~7% increase in throughput and ~9-10% decrease in latency for E2E inference workloads, while maintaining acceptable numerical accuracy.