This feature thread tracks the evolution of enabling massive-scale UMAP (Uniform Manifold Approximation and Projection) for dimensionality reduction on datasets with tens to hundreds of millions of vectors, leveraging multi-GPU parallelism. Initial efforts focused on single-GPU UMAP and out-of-core approaches for datasets exceeding GPU memory. Subsequent developments introduced multi-GPU support for the critical all-neighbors kNN graph construction step, allowing for end-to-end scaling and dramatic reductions in training runtimes. This includes the introduction of parameters like `knn_n_clusters` and `knn_overlap_factor` to manage the trade-off between space, time, and quality, and the integration with the cuVS all-neighbors API for standalone graph construction.
All NVIDIA trails