# LLM Benchmarks Tracker > LLM Benchmarks Tracker is a sourced, schema-validated catalogue of 54 LLM and agent evaluation benchmarks (31 model, 23 agent) and 18 evaluators, in which every score records who published it, under which conditions, and what kind of source it came from. Most benchmark round-ups copy vendor slide numbers with no provenance. Here each of the 252 result rows carries the URL it was published at, the source kind (official leaderboard, paper, independent evaluation, developer self-report, aggregator), the access date, and the evaluation conditions the source stated (split, tools, reasoning effort, scaffold, pass@k); 52% of current top scores come from an official leaderboard, a paper or an independent evaluation. Benchmarks also carry a saturation status and a contamination risk, so a stale benchmark can be recognised as stale. This project aggregates and labels third-party results; it runs no evaluations of its own. Read the data without installing anything: `curl https://alloevil.github.io/llm-benchmarks-tracker/api/v1/benchmarks.json`. To work on it: `git clone https://github.com/alloevil/llm-benchmarks-tracker && cd llm-benchmarks-tracker && pip install -e ".[dev]"`. ## Docs - [Benchmark catalogue](https://alloevil.github.io/llm-benchmarks-tracker/): every benchmark with its top sourced score, status and contamination risk - [Chinese edition](https://alloevil.github.io/llm-benchmarks-tracker/zh/): the same data with Simplified Chinese descriptions - [Contributing guide](https://github.com/alloevil/llm-benchmarks-tracker/blob/main/CONTRIBUTING.md): what a result row must carry to be accepted - [Changelog](https://github.com/alloevil/llm-benchmarks-tracker/blob/main/CHANGELOG.md): dated record of catalogue changes ## Data - [JSON API index](https://alloevil.github.io/llm-benchmarks-tracker/api/v1/index.json): every endpoint, generated with the site - [Benchmarks with top scores](https://alloevil.github.io/llm-benchmarks-tracker/api/v1/benchmarks.json): metadata plus sota and result_count per benchmark - [Evaluators](https://alloevil.github.io/llm-benchmarks-tracker/api/v1/evaluators.json): frameworks, leaderboards, independent evaluators, aggregators - [Per-benchmark ledger](https://alloevil.github.io/llm-benchmarks-tracker/api/v1/results/swe-bench-verified.json): one file per benchmark under api/v1/results/ - [Benchmark schema](https://alloevil.github.io/llm-benchmarks-tracker/schema/benchmark.schema.json): the contract every benchmark file is validated against - [Results schema](https://alloevil.github.io/llm-benchmarks-tracker/schema/results.schema.json): requires a source URL, source kind and access date per row ## Evidence - [claims.json](https://alloevil.github.io/llm-benchmarks-tracker/claims.json): every number on this site with its metric, method, repro command, executable check and evidence - [llms-full.txt](https://alloevil.github.io/llm-benchmarks-tracker/llms-full.txt): self-contained description, install, limits and FAQ ## Benchmarks - [BenchCAD](https://alloevil.github.io/llm-benchmarks-tracker/b/benchcad/): model benchmark, multimodal, code, reasoning; active, contamination risk medium; top 0.959 (aggregator) - [AA-Omniscience](https://alloevil.github.io/llm-benchmarks-tracker/b/aa-omniscience/): model benchmark, knowledge, factuality; active, contamination risk low; top 44 (independent) - [GDPval](https://alloevil.github.io/llm-benchmarks-tracker/b/gdpval/): model benchmark, general-assistant, knowledge, instruction-following; active, contamination risk medium; top 74.1% (self-reported) - [HealthBench](https://alloevil.github.io/llm-benchmarks-tracker/b/healthbench/): model benchmark, knowledge, safety, instruction-following; active, contamination risk medium; top 59.9% (paper) - [ARC-AGI-2](https://alloevil.github.io/llm-benchmarks-tracker/b/arc-agi-2/): model benchmark, reasoning; saturating, contamination risk medium; top 95% (official) - [ScreenSpot-Pro](https://alloevil.github.io/llm-benchmarks-tracker/b/screenspot-pro/): model benchmark, computer-use, multimodal; saturating, contamination risk high; top 92.7% (self-reported) - [Humanity's Last Exam](https://alloevil.github.io/llm-benchmarks-tracker/b/hle/): model benchmark, knowledge, reasoning, science, math, multimodal; active, contamination risk medium; top 65% (self-reported) - [Aider Polyglot](https://alloevil.github.io/llm-benchmarks-tracker/b/aider-polyglot/): model benchmark, code, instruction-following; saturating, contamination risk medium; top 88% (official) - [SimpleQA](https://alloevil.github.io/llm-benchmarks-tracker/b/simpleqa/): model benchmark, factuality, knowledge; active, contamination risk high; top 62.5% (self-reported) - [FrontierMath](https://alloevil.github.io/llm-benchmarks-tracker/b/frontiermath/): model benchmark, math, reasoning, research; saturating, contamination risk low; top 93.7% (independent) - [MMMU-Pro](https://alloevil.github.io/llm-benchmarks-tracker/b/mmmu-pro/): model benchmark, multimodal, knowledge, reasoning; active, contamination risk medium; top 86.9% (self-reported) - [MMLU-Pro](https://alloevil.github.io/llm-benchmarks-tracker/b/mmlu-pro/): model benchmark, knowledge, reasoning; saturating, contamination risk high; top 91% (independent) - [RULER](https://alloevil.github.io/llm-benchmarks-tracker/b/ruler/): model benchmark, long-context; saturating, contamination risk low; top 95.1% (official) - [LiveCodeBench](https://alloevil.github.io/llm-benchmarks-tracker/b/livecodebench/): model benchmark, code, reasoning; active, contamination risk low; top 93.5% (self-reported) - [BFCL](https://alloevil.github.io/llm-benchmarks-tracker/b/bfcl/): model benchmark, tool-use; active, contamination risk medium; top 77.47% (official) - [MMMU](https://alloevil.github.io/llm-benchmarks-tracker/b/mmmu/): model benchmark, multimodal, knowledge, reasoning; saturating, contamination risk high; top 85.4% (self-reported) - [IFEval](https://alloevil.github.io/llm-benchmarks-tracker/b/ifeval/): model benchmark, instruction-following; saturating, contamination risk high; top 95% (self-reported) - [GPQA Diamond](https://alloevil.github.io/llm-benchmarks-tracker/b/gpqa-diamond/): model benchmark, science, reasoning, knowledge; saturating, contamination risk medium; top 96% (aggregator) - [LMArena Text](https://alloevil.github.io/llm-benchmarks-tracker/b/lmarena-text/): model benchmark, human-preference, general-assistant; active, contamination risk low; top 1466 (official) - [OSWorld 2.0](https://alloevil.github.io/llm-benchmarks-tracker/b/osworld-2/): agent benchmark, computer-use, multimodal, tool-use; active, contamination risk medium; top 72.6% (self-reported) - [AutomationBench](https://alloevil.github.io/llm-benchmarks-tracker/b/automationbench/): agent benchmark, tool-use, general-assistant, instruction-following; active, contamination risk low; top 50.3% (official) - [ARC-AGI-3](https://alloevil.github.io/llm-benchmarks-tracker/b/arc-agi-3/): agent benchmark, reasoning, tool-use; saturating, contamination risk low; top 99.9% (official) - [Terminal-Bench](https://alloevil.github.io/llm-benchmarks-tracker/b/terminal-bench/): agent benchmark, software-engineering, code, tool-use, ml-engineering; active, contamination risk medium; top 64.6% (self-reported) - [DeepSearchQA](https://alloevil.github.io/llm-benchmarks-tracker/b/deepsearchqa/): agent benchmark, web, research, factuality, tool-use; active, contamination risk medium; top 95% (aggregator) - [Tool Decathlon (Toolathlon)](https://alloevil.github.io/llm-benchmarks-tracker/b/tool-decathlon/): agent benchmark, tool-use, general-assistant, software-engineering; active, contamination risk medium; top 78.4% (official) - [SWE-Bench Pro](https://alloevil.github.io/llm-benchmarks-tracker/b/swe-bench-pro/): agent benchmark, software-engineering, code, tool-use; active, contamination risk medium; top 61.5% (official) - [tau2-bench](https://alloevil.github.io/llm-benchmarks-tracker/b/tau2-bench/): agent benchmark, tool-use, instruction-following, general-assistant; active, contamination risk medium; top 87.9% (official) - [FieldWorkArena](https://alloevil.github.io/llm-benchmarks-tracker/b/fieldworkarena/): agent benchmark, multimodal, general-assistant, safety, reasoning; active, contamination risk low; top 52% (paper) - [PaperBench](https://alloevil.github.io/llm-benchmarks-tracker/b/paperbench/): agent benchmark, research, ml-engineering, code, tool-use; active, contamination risk medium; top 43.4% (official) - [BrowseComp](https://alloevil.github.io/llm-benchmarks-tracker/b/browsecomp/): agent benchmark, web, tool-use, factuality, reasoning; saturating, contamination risk medium; top 92.2% (aggregator) - [SWE-bench Multilingual](https://alloevil.github.io/llm-benchmarks-tracker/b/swe-bench-multilingual/): agent benchmark, software-engineering, code, tool-use; active, contamination risk high; top 72.7% (official) - [MLE-bench](https://alloevil.github.io/llm-benchmarks-tracker/b/mle-bench/): agent benchmark, ml-engineering, code, tool-use; active, contamination risk medium; top 64.44% (official) - [SWE-bench Verified](https://alloevil.github.io/llm-benchmarks-tracker/b/swe-bench-verified/): agent benchmark, software-engineering, code, tool-use; saturating, contamination risk high; top 96% (aggregator) - [OSWorld](https://alloevil.github.io/llm-benchmarks-tracker/b/osworld/): agent benchmark, computer-use, multimodal, tool-use; active, contamination risk medium; top 90.19% (official) - [VisualWebArena](https://alloevil.github.io/llm-benchmarks-tracker/b/visualwebarena/): agent benchmark, web, multimodal, computer-use; active, contamination risk high; top 54% (official) - [GAIA](https://alloevil.github.io/llm-benchmarks-tracker/b/gaia/): agent benchmark, general-assistant, tool-use, web, reasoning, multimodal; saturating, contamination risk medium; top 93.36% (official) - [WebArena](https://alloevil.github.io/llm-benchmarks-tracker/b/webarena/): agent benchmark, web, tool-use, computer-use; active, contamination risk high; top 74.3% (official) ## Source - [GitHub repository](https://github.com/alloevil/llm-benchmarks-tracker)