IFEval
Instruction-Following Evaluation for Large Language Models
541 prompts each containing one or more of 25 types of verifiable instructions, such as 'write at least 400 words', 'respond in JSON', 'include the keyword X three times' or 'no commas'. Compliance is checked programmatically, giving prompt-level and instruction-level accuracy under strict and loose matching. Cheap, reproducible and free of LLM judges, it is the standard check that a chat model actually obeys formatting constraints.
- Released
- 2023-11
- Maintainer
- Google (Zhou et al.)
- Status
- Contamination
- high
- Metric
- prompt-level strict accuracy (percent, ↑)
- Tasks
- 541
- Domains
- instruction-following
- human
- no measured baseline
Notes. Vendors frequently report instruction-level or loose accuracy without saying so, which inflates numbers by 5-10 points; check the variant before comparing. Most frontier models exceed 90% prompt-level strict, and IFBench and IFEval-Hard have emerged as harder follow-ups.
Full ledger
| System | Developer | Score | Date | Source | Conditions |
|---|---|---|---|---|---|
| Qwen3.5-27B | Alibaba | 95% | self-reported | Model card benchmark table; same table lists GPT-5-mini (2025-08-07) at 93.9 as run by Qwen. Variant not stated. Also the top row on benchlm.ai's IFEval page. | |
| Qwen3-235B-A22B-Instruct-2507 | Alibaba | 88.7% | self-reported | Model card 'Alignment' table; the same table lists Kimi K2 at 89.8, Claude Opus 4 (non-thinking) 87.4 and GPT-4o-0327 83.9 as run by Qwen. Variant not stated. | |
| Llama 3.1 405B Instruct | Meta | 88.6% | self-reported | Llama 3.1 model card benchmark table (70B: 87.5, 8B: 80.4); Meta does not state which IFEval variant. Date is the Llama 3.1 release. | |
| GPT-4 | OpenAI | 76.89% | paper | split: prompt-level strict Table 3 of the IFEval paper (responses collected November 2023); instruction-level strict 83.57, loose 79.30/85.37. |