NVIDIA Releases Kumo Tabular, an Open Foundation Model for Tabular Data

NVIDIA Introduces Open Tabular Foundation Model Kumo Tabular

NVIDIA researchers announced the release of NVIDIA Kumo Tabular, an open foundation model designed for tabular data classification and regression tasks. Part of the NVIDIA Kumo Structured collection, the model predicts labels for new rows in a single forward pass without requiring manual feature engineering, hyperparameter search, or task-specific training.

Kumo Tabular comes in three sizes ranging from 28 million to 215 million parameters and is released alongside an open-source library under the OpenMDW-1.1 license for commercial use. Model weights have been made available on Hugging Face, while the underlying code is hosted on GitHub under NVIDIA's structured data repository.

Architecture, Synthetic Pretraining, and Benchmark Results

Built as a Transformer tailored to tabular structures, Kumo Tabular incorporates column, row, and in-context attention mechanisms drawing on concepts from TabICL and TabPFN. The pipeline begins with cell embedding, where groups of cells become tokens; numerical and categorical values pass through Fourier features with learned frequencies, and missing values are accommodated directly without requiring prior imputation. Row embeddings alternate between column attention—which tracks feature distribution down columns with cost scaling linearly with row count—and row attention across features using rotary position embeddings and four learnable [CLS] tokens.

The final stage relies on in-context learning where context rows attend to each other, but query rows attend strictly to context rows via Test-GQA, allowing cached keys and values to be reused across future predictions. To prevent attention degradation over larger tables, the model introduces length-aware attention temperature, scaling queries logarithmically with key counts using head-specific learned coefficients. Regression heads output 999 quantiles alongside point estimates and uncertainty measures.

Kumo Tabular was trained entirely on procedurally generated synthetic tables derived from Structural Causal Models (SCMs). To reflect real-world messiness, the procedural sampler injects missing value patterns, heavy-tailed regression targets, high-cardinality categories, and coarsened features. Training was structured across three stages: starting with 1,024 rows and up to 100 columns, expanding to between 400 and 10,240 rows, and ultimately extending up to 60,000 rows. In total, the Small, Medium, and Large variants observed approximately 35 million, 71 million, and 137 million artificial tables, respectively.

Across empirical benchmarks:

• TabArena: Kumo Tabular took the first-place overall ranking with an ELO of 1950, clocking in 17 times faster than LimiX-2 on an evaluation setup powered by a single NVIDIA RTX 6000 Pro.
• BeyondArena: It ranked first with an ELO of 1418 and an Improvability score of 7.78%.
• TALENT: The model achieved the top overall ranking, scoring average ranks of 6.67 in classification accuracy, 3.98 in classification log-loss, and 4.22 in regression RMSE.
• ScoringBench: The Large and Medium checkpoints ranked first and second in average rank.

The model natively processes numerical and categorical inputs for up to 10 classes in a single forward pass, with NVIDIA's GPU-native library (sdm) utilizing error-correcting output codes to extend support to arbitrary class counts. NVIDIA notes that performance can degrade if query distributions diverge sharply from context rows or fall outside training ranges.

Shifting Enterprise Tabular Workflows Toward In-Context Learning

For roughly two decades, enterprise tabular prediction tasks—such as customer churn, fraud, pricing, and default risk—have relied on gradient-boosted decision trees and AutoML frameworks like AutoGluon. While effective, traditional tree workflows require training dedicated models from scratch for every separate problem and feature set.

Kumo Tabular adapts the in-context learning paradigm popularized by Large Language Models to structured data tables. By reading existing labeled examples directly as context within the prompt space, tabular foundation models aim to replace the cycle of manual feature preprocessing, hyperparameter optimization, and separate deployment pipelines with prompt-based inference in a single forward execution.