ARC-AGI-2
Abstraction and Reasoning Corpus for Artificial General Intelligence, version 2
Second-generation ARC grid puzzles designed to defeat brute-force search and test symbolic interpretation, compositional reasoning and context-dependent rule application. 1,000 public training tasks plus calibrated 120-task public, semi-private and private evaluation sets; every task was solved by at least two humans within two attempts. Scored as the percentage of test outputs exactly correct within two attempts, always reported together with cost per task. It launched with ARC Prize 2025 as the successor to ARC-AGI-1.
paper · website · leaderboard · dataset · code
- Released
- 2025-03
- Maintainer
- ARC Prize Foundation
- Status
- Contamination
- medium
- Metric
- accuracy (pass@2) (percent, ↑)
- Tasks
- 120
- Domains
- reasoning
- human
- 100% ARC Prize human panel (every task solved by at least two of 400+ general-public testers); 66% of individual attempts succeeded
Notes. Announced 2025-03-24; the arXiv paper followed in May 2025. The ARC Prize grand-prize threshold is 85% on the private set under Kaggle compute limits, which no open solution has met. Frontier API models passed 90% on the Semi-Private set in mid-2026 at several dollars per task, so the benchmark is saturating for unconstrained systems while remaining open for efficient ones.
Full ledger
| System | Developer | Score | Date | Source | Conditions |
|---|---|---|---|---|---|
| GPT-6 Astra (Max) | OpenAI | 95% | official | split: Semi-Private pass_k: 2 cost_usd_per_task: 1.12 ARC Prize verified; also republished by benchlm.ai and in OpenAI's GPT-6 Astra launch table (95.0%). | |
| GPT-5.6 Sol (Max) | OpenAI | 92.5% | official | split: Semi-Private pass_k: 2 cost_usd_per_task: 1.44 | |
| Gemini 3 Deep Think (2/26) | Google DeepMind | 84.6% | official | split: Semi-Private pass_k: 2 cost_usd_per_task: 13.62 | |
| GPT-5.2 (XHigh) | OpenAI | 52.9% | official | split: Semi-Private pass_k: 2 cost_usd_per_task: 1.9 | |
| o3 (Medium) | OpenAI | 3% | paper | split: Semi-Private pass_k: 2 Table 1 of the ARC-AGI-2 paper (scores as of 14 May 2025); the same model scored 53% on ARC-AGI-1. |