Abstract
Tabular foundation models show varying performance across different data conditions, with traditional methods still outperforming newer approaches on complex, large-scale datasets.
Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry. Research communities across disciplines are increasingly evaluating tabular foundation models on diverse datasets and tasks. However, these task- and discipline-specific evaluations remain largely inaccessible to model researchers because benchmark software and evaluation protocols are fragmented. As a result, model researchers rely on standard benchmarks, which are mostly defined for tasks where tabular foundation models already excel. The most challenging scenarios are excluded, limiting meaningful progress in the field by focusing on marginal improvements on IID data rather than on broader, more demanding challenges. To overcome this, we introduce BeyondArena, the first unified holistic benchmark for tabular data that supports diverse task types (IID, temporal, grouped), across sample size and feature dimensionality scales, with diverse feature types (with text, with high cardinality) from a broad range of disciplines. To enable unified benchmarking beyond standard benchmarks, we introduce Data Foundry, a Python framework and metadata schema for curating tabular datasets for predictive machine learning. Our results across 11 models and 142 curated datasets show that existing tabular foundation models excel on tiny- to medium-sized IID data, while traditional tree-based and deep learning models still dominate on non-IID, large, and high-dimensional datasets. BeyondArena guides model research for the most demanding challenges in tabular data, enabling progress towards truly foundational tabular models.
Community
How General Are Tabular Foundation Models, Really? š¤Ø
Our new benchmark, BeyondArena, from our paper "Beyond IID: How General Are Tabular Foundation Models, Really?" shows across 142 curated datasets:
ā Tabular foundation models excel on tiny-to-medium IID dataš„
ā Tree-based and deep learning models still dominate on non-IID, large, and high-dimensional dataā
BeyondArena extends TabArena-v0.1 from only small-to-medium IID data to the hard cases (temporal splits, grouped data, very large or very high-dimensional tables, free-text columns, high-cardinality features). BeyondArena allows model developers to tackle the challenges that would make a model truly foundational. It spans:
- Diverse tabular task types: IID, non-IID temporal, and non-IID grouped
- A wide range of dataset sizes and feature dimensionalities (from 100 to 1 million rows, from 3 to 22k features)
- Hard feature types: free text and high-cardinality categorical
- Datasets drawn from a broad range of disciplines
To make this kind of benchmarking rigorous, we also release Data Foundry, a Python framework and metadata schema for curating tabular datasets for predictive ML.
How to engage with BeyondArena?
- Read the Paper: https://arxiv.org/abs/2606.30410
- Run a benchmark with our code: https://tabarena.ai/code
- Curate a new dataset with Data Foundry: https://github.com/TabArena/data-foundry
- Check out our dataset curation notes: https://tabarena.github.io/data-foundry/
- Investigate our datasets: https://huggingface.co/datasets/TabArena/BeyondArena
Relation to TabArena: BeyondArena is our research work towards TabArena-v0.2, the next generation of tabular benchmarking. Right now, BeyondArena is already fully integrated into the TabArena ecosystem! We will go from BeyondArena to TabArena-v0.2 after adding more datasets and baselines. Our recommendation is to ensure your model works well on TabArena-v0.1, then try it on BeyondArena!
Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper