Nums AI's Causilo tops TabArena single-model leaderboard

Nums AI released Causilo yesterday, a tabular foundation model that sits at the top of TabArena's single-model rankings for both classification and regression with an Elo of 1,794. The model ships with a scikit-learn interface, Apache-2.0-licensed code, and pretrained weights on Hugging Face, but commercial, production, or hosted-API use requires a separate license from the company.

How it works: in-context learning without weight updates

Causilo is an in-context learner. A call to fit does not update the pretrained weights; it stores the training rows as context and predicts query rows in a single forward pass. Nums says pretraining used only synthetic data — none of the TabArena datasets. Inputs can be NumPy arrays or pandas DataFrames, including categorical features and missing values. Classification supports up to 10 classes; regression returns the mean by default, and version 1.0.1 adds median and quantiles derived from 999 original quantiles.

Architecture: refinement, compression, and in-context learning

The company splits the network into three stages: refinement, compression, and in-context learning. Features are grouped in threes; each value receives an embedding with 16 learned sine and cosine frequencies, and missing values get their own learned vector. Two column stages summarize each feature group through 128 latent slots that read only the training rows. Between them, a row stage enables cross-group interaction via 4 latent tokens using cross-attention instead of full self-attention, which Nums says keeps cost linear in the number of features. A pooling block compresses each row to a fixed 512-dimensional vector, labels are appended to the training rows, and a 12-layer prediction block lets query rows attend to those same labeled rows without modifying the context or each other. The default runs 8 ensemble members that share weights, each passing through a different normalization with seeded permutations.

TabArena results: first place with an asterisk

The official run covered 51 datasets and 816 Full splits with 8 estimators and seed 42; the benchmark maintainer re-ran the full evaluation and confirmed the same overall Elo of 1,794. Notable rivals: Google Research's TabFM, LG AI Research's EXAONE Tabular, and Prior Labs' TabPFN-3 (1,636.2). Caveats: the top spot excludes system entries — with them included, Causilo ranks third out of 88 total. On improvability, TabFM still leads overall and in classification, while Causilo leads in regression. Elo confidence intervals at the top overlap, so the edge over TabFM and EXAONE is narrow. Nums also cites Xiaomi-TabLDM and Amazon's Mitra-v2 behind Causilo, though neither appears in the benchmark files in the model repository.

ScoringBench and speed: leads on calibration metrics too

On ScoringBench, which evaluates regression models with proper scoring rules such as CRPS alongside RMSE and R², Nums submitted Causilo 1.0.1 across 101 datasets, 5 folds each, with a 3,000-sample cap. The company reports first place in CRPS, R², and RMSE, and the benchmark maintainer independently verified the results before approval. In a speed-and-memory test on a single H100 with 80 GB and 8 CPU cores, Causilo was fastest in both fit and predict; TabPFN-3 consumed far less GPU memory. Setting use_kv_cache=True moves the context work to the fit stage, trading higher memory use for faster repeated predictions.

Requirements and install: Python 3.10+, PyTorch 2.13

Causilo requires Python 3.10 through 3.12 and PyTorch 2.13 or later. The first fit downloads the checkpoint automatically. Installation is straightforward: pip install causilo, then import CausiloClassifier or CausiloRegressor with the desired parameters.