NVIDIA Kumo Tabular Sets a New Accuracy-Efficiency Frontier for Tabular Prediction

This post introduces NVIDIA Kumo Tabular, a new foundation model for tabular data prediction. It details the model's architecture, which includes cell embedding (using Fourier features, sines/cosines, and label embeddings), row embedding (alternating column and row attention), and in-context learning (using Test-GQA and length-aware attention temperature). The model achieves state-of-the-art performance on several tabular benchmarks and is released under the OpenMDW-1.1 license.