MLE-bench
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
75 个 Kaggle 竞赛(22 低、38 中、15 高复杂度),智能体需在沙盒内于算力预算之内阅读任务、准备数据、训练模型并提交预测。得分为提交结果在真实 Kaggle 排行榜上至少能获得铜牌的竞赛比例。是端到端 ML 工程(而非孤立编码)的标准测试。
- 发布
- 2024-10
- 维护者
- OpenAI (Chan et al.)
- 状态
- active
- 污染风险
- medium
- 指标
- any-medal rate (percent, ↑)
- 题量
- 75
- 领域
- ml-engineering code tool-use
- 人类
- 无实测基线
备注. Competitions are public Kaggle data and the paper documents contamination checks. Leaderboard rows are self-submitted with grading reports; as of April 2026 OpenAI paused new submissions while redesigning the process. Runs use 24h (sometimes 12h/36h) budgets, which affects comparability.
完整账本
| 系统 | 开发者 | 分数 | 日期 | 来源 | 条件 |
|---|---|---|---|---|---|
| Famou-Agent 2.0 + Gemini-3-Pro-Preview | Baidu | 64.44% | 官方榜单 | split: all scaffold: Famou-Agent 2.0 Top of the main leaderboard as of access date; 24h budget. | |
| AIBuildAI + Claude-Opus-4.6 | AIBuildAI | 63.11% | 官方榜单 | split: all scaffold: AIBuildAI 24h budget. | |
| Famou-Agent + Gemini-2.5-Pro | Baidu | 43.56% | 官方榜单 | split: all scaffold: Famou-Agent 24h budget. | |
| AIRA-dojo + o3 | Meta FAIR | 31.6% | 官方榜单 | split: all scaffold: AIRA-dojo 24h budget. | |
| AIDE + o1-preview | OpenAI | 17.12% | 官方榜单 | split: all scaffold: AIDE Paper-era baseline (16.9% in abstract, 17.12 +/- 0.61 on the leaderboard); 24h budget. |