Leaderboard Forensics Series' Articles

Cover image for When a 0.4-Point Lead Means Nothing: Reading Open-Model Leaderboard Margins Like a Statistician

Ward Ed

Oct 6

When a 0.4-Point Lead Means Nothing: Reading Open-Model Leaderboard Margins Like a Statistician

#huggingface #llm #benchmark #machinelearning

7 min read

Cover image for Benchmark Contamination 101: How Train/Test Overlap Inflates Leaderboard Scores (and How to Catch It)

Ward Ed

Oct 6

Benchmark Contamination 101: How Train/Test Overlap Inflates Leaderboard Scores (and How to Catch It)

#machinelearning #llm #benchmark #datascience

1

3

7 min read

Ward Ed

Oct 12

How to Read a Public Leaderboard Without Fooling Yourself

#ai #machinelearning #benchmark #datascience

3 min read

Cover image for Best Open-Weight LLM in October 2026? Auditing Two Leaderboards Row by Row

Ward Ed

Oct 19

Best Open-Weight LLM in October 2026? Auditing Two Leaderboards Row by Row

#ai #machinelearning #llm #benchmark

1

12 min read

Cover image for Do 66 Terminal Tasks and a 45% Private Weight Settle AI Rankings? Reading Artificial Analysis Index v4.3

Ward Ed

Oct 20

Do 66 Terminal Tasks and a 45% Private Weight Settle AI Rankings? Reading Artificial Analysis Index v4.3

#ai #machinelearning #llm #benchmark

12 min read

Cover image for GPQA Diamond Has 198 Questions: What the Hugging Face Leaderboard's 115 Rows Can and Cannot Rank

Ward Ed

Oct 21

GPQA Diamond Has 198 Questions: What the Hugging Face Leaderboard's 115 Rows Can and Cannot Rank

#ai #opensource #llm #benchmark

14 min read

Cover image for Reading a Hardware-Equivalence Claim Like an Evaluator: A Checklist Applied to an NPU vs GPU Pretraining Twin

Ward Ed

Oct 26

Reading a Hardware-Equivalence Claim Like an Evaluator: A Checklist Applied to an NPU vs GPU Pretraining Twin

#ai #machinelearning #benchmark #llm

1

2 min read

Cover image for SWE-Bench Pro Has Two Versions: What the Hugging Face Leaderboard's 47 Rows Mix Together

Ward Ed

Oct 31

SWE-Bench Pro Has Two Versions: What the Hugging Face Leaderboard's 47 Rows Mix Together

#ai #opensource #llm #benchmark

14 min read

Cover image for Humanity's Last Exam on Hugging Face: At Least 23 of the Top 24 Rows Use Tools, and the Board Has No Column for It

Ward Ed

Nov 1

Humanity's Last Exam on Hugging Face: At Least 23 of the Top 24 Rows Use Tools, and the Board Has No Column for It

#ai #machinelearning #llm #benchmark

15 min read

Cover image for Terminal-Bench 2.0 vs 2.1 on Hugging Face: A 2.1 Score Is Ranked First on the 2.0 Board

Ward Ed

Nov 2

Terminal-Bench 2.0 vs 2.1 on Hugging Face: A 2.1 Score Is Ranked First on the 2.0 Board

#ai #machinelearning #llm #benchmark

15 min read

$Cover image for GSM8K on Hugging Face: 4 of the Top 12 Rows Show a Sibling Checkpoint's Score, and 20 of 28 Rows Are Unmerged PRs$

Ward Ed

Nov 3

GSM8K on Hugging Face: 4 of the Top 12 Rows Show a Sibling Checkpoint's Score, and 20 of 28 Rows Are Unmerged PRs

#ai #machinelearning #llm #benchmark

14 min read

Cover image for SWE-bench Verified on Hugging Face: 9 Self-Reported Rows Top the Official Table's Best, and 6 Repos Fill 16 of 78 Rows

Ward Ed

Nov 5

SWE-bench Verified on Hugging Face: 9 Self-Reported Rows Top the Official Table's Best, and 6 Repos Fill 16 of 78 Rows

#ai #machinelearning #llm #benchmark

1

14 min read

Cover image for Open ASR Leaderboard on Hugging Face: Ranks 1 and 2 Swap on a Common Mean, and 8 of 65 Rows Skip TED-LIUM

Ward Ed

Nov 7

Open ASR Leaderboard on Hugging Face: Ranks 1 and 2 Swap on a Common Mean, and 8 of 65 Rows Skip TED-LIUM

#ai #machinelearning #speechrecognition #benchmark

1

16 min read

Cover image for olmOCR-bench on Hugging Face: Three Rows Use a Different Average, and One Moves From 13th to 7th or 19th

Ward Ed

Nov 8

olmOCR-bench on Hugging Face: Three Rows Use a Different Average, and One Moves From 13th to 7th or 19th

#ai #machinelearning #ocr #benchmark

14 min read

Cover image for SWE-bench Multilingual on Hugging Face: 15 of 34 Scores Are Not a Count Out of 300 Tasks

Ward Ed

Nov 9

SWE-bench Multilingual on Hugging Face: 15 of 34 Scores Are Not a Count Out of 300 Tasks

#ai #machinelearning #benchmark #llm

15 min read

Cover image for WildClawBench on Hugging Face: Rank 1 Is a Runtime, Because Scores, Minutes and Dollars Share One Ranked Column

Ward Ed

Nov 11

WildClawBench on Hugging Face: Rank 1 Is a Runtime, Because Scores, Minutes and Dollars Share One Ranked Column

#ai #machinelearning #benchmark #agents

17 min read