Largest accuracy drop among tested model families on GSM1k, a fresh human-written mirror of the GSM8K math benchmark
up to 8 ptsobserved
| Value kind | observed — Reported measurements, counts, and specifications keep the precision and scope stated by their source; an exact specification is not treated as a range. |
|---|---|
| Scope | The largest drop among the model families tested, not an average and not a contamination rate. Frontier models showed minimal signs of overfitting. |
| Caveat | The first arXiv version of the paper reported drops of up to 13%; the figure here is from the final version (NeurIPS 2024 Datasets and Benchmarks). |
| As of | 2024 |
| Source | Zhang et al. (Scale AI), 'A Careful Examination of Large Language Model Performance on Grade School Arithmetic', NeurIPS 2024 Datasets and Benchmarks |
| Review | checking…review by 2027-10-16 · standard cadence |
| Recorded changes | last 2026-10-16 |
| Claim id | gsm1k-clean-mirror-accuracy-drop-vs-gsm8k |
Where the guide uses it
← Full numbers register — every date-stamped figure in the guide, with revision history.