IFEval
Instruction-Following Evaluation for Large Language Models
541 个提示,每个包含 25 类可验证指令中的一种或多种,例如"至少写 400 字"、"以 JSON 回复"、"包含关键词 X 三次"或"不使用逗号"。合规性通过程序检查,给出严格与宽松匹配下的提示级和指令级准确率。低成本、可复现且不依赖 LLM 评判,是检验聊天模型是否真正遵守格式约束的标准。
- 发布
- 2023-11
- 维护者
- Google (Zhou et al.)
- 状态
- 污染风险
- high
- 指标
- prompt-level strict accuracy (percent, ↑)
- 题量
- 541
- 领域
- instruction-following
- 人类
- 无实测基线
备注. Vendors frequently report instruction-level or loose accuracy without saying so, which inflates numbers by 5-10 points; check the variant before comparing. Most frontier models exceed 90% prompt-level strict, and IFBench and IFEval-Hard have emerged as harder follow-ups.
完整账本
| 系统 | 开发者 | 分数 | 日期 | 来源 | 条件 |
|---|---|---|---|---|---|
| Qwen3.5-27B | Alibaba | 95% | 厂商自报 | Model card benchmark table; same table lists GPT-5-mini (2025-08-07) at 93.9 as run by Qwen. Variant not stated. Also the top row on benchlm.ai's IFEval page. | |
| Qwen3-235B-A22B-Instruct-2507 | Alibaba | 88.7% | 厂商自报 | Model card 'Alignment' table; the same table lists Kimi K2 at 89.8, Claude Opus 4 (non-thinking) 87.4 and GPT-4o-0327 83.9 as run by Qwen. Variant not stated. | |
| Llama 3.1 405B Instruct | Meta | 88.6% | 厂商自报 | Llama 3.1 model card benchmark table (70B: 87.5, 8B: 80.4); Meta does not state which IFEval variant. Date is the Llama 3.1 release. | |
| GPT-4 | OpenAI | 76.89% | 论文 | split: prompt-level strict Table 3 of the IFEval paper (responses collected November 2023); instruction-level strict 83.57, loose 79.30/85.37. |