Why Large Language Models Fail at Tabular Prediction

Garnelo and Czarnecki systematically investigate why LLMs fail at in-context tabular prediction — one of the most common ML workloads — using controlled experiments that falsify four of five popular explanations before isolating one decisive factor: dimensionality.

They test Claude Opus 4.6 in "pure inference mode" (single prompt with full training and test data, no tools, no scaffolding, no fine-tuning) and evaluate five hypotheses drawn from practitioner folklore. H1 (class overlap/non-separability), H2 (linearised CSV format obscuring column structure), H3 (numeric tokenisation/precision), and H4 (per-query test load) are all rejected by targeted interventions. H5 (dimensionality) is the only one that survives: in a sweep of random linear projections over 31 benchmark datasets, the LLM is the only method among nine whose accuracy decreases as dimensionality grows, while every classical baseline stays flat or improves.

A memorisation probe first cleans the benchmark: the model recovers original labels for breast cancer, iris, and wine almost perfectly — direct evidence of dataset contamination — and these are excluded from downstream analysis.

On 2D tasks where the model does work, its decision boundaries are reproduced almost exactly by distance-based methods (Gaussian processes and low-k NN, 91–92% grid agreement). In higher dimensions, however, no classical model among 252 configured baselines exceeds 64.8% prediction agreement, and adding dimension-dependent label noise to every baseline improves agreement by at most 0.64 percentage points. The LLM's high-dimensional predictions are not those of any standard learner, nor of such a learner degraded to random guessing — the mechanism of the collapse remains open.

Reasoning traces are also examined: in 14 of 20 2D tasks, the model's verbal explanations fail to match either its predictions or the ground truth, consistent with known faithfulness gaps between stated reasoning and actual computation.

The practical implication: no amount of prompt engineering aimed at format, precision, or batching will fix high-dimensional in-context tabular prediction. Purpose-built tabular foundation models remain the right tool.