Tabular Foundation Models

NVIDIA Kumo Tabular, an open foundation model for tabular data, is now available. It enables in-context learning for tabular prediction tasks (classification and regression) without training or feature engineering. The model is pretrained on artificial data, comes in various sizes, and ranks first on multiple benchmarks. It utilizes a Transformer architecture with cell, row, and in-context attention mechanisms, including Test-GQA for efficient query processing and length-aware attention temperature.