Speeding Up AI: Bringing Google Colossus to PyTorch via GCSFS and Rapid Bucket- Google Developers Blog

Introduced Rapid Bucket, a high-performance object storage solution leveraging Google's Colossus architecture and gRPC bidirectional streams for AI/ML workloads. Integrated Rapid Storage with PyTorch via `fsspec` and `gcsfs`, enabling significant performance gains (23% reduction in training time, 4.8x read throughput, 2.8x write throughput) by bypassing legacy REST APIs and utilizing zonal co-location for reduced latency. This integration requires no code rewrites for existing `fsspec` clients.