Artificial Analysis Rebuilt Its Index. The Open-Weight Gap Went From 3 Points to 12, and 15% of the Score Is a Private Cognition Benchmark.
Artificial Analysis shipped Intelligence Index v5 on October 27: nine evaluations, 60% of them private or held-out, with Cognition's private FrontierCode 2.0 alone worth 15% of the score. Claude Opus 5.5 leads at 58, GPT-6 Astra 57, GPT-6.1 Sol 55, Gemini 4 Argon 50. Kimi K3.1 Pro scores 46 with weights still unreleased; the best released open weights, DeepSeek V4.1 Flash 1017 and GLM-5.3, sit at 32, below Claude Haiku 5.5 at 36. Every v4.3.2 score on this site, including our own Index page, is now on the old scale. We read AA's launch article and methodology page.
By T.W. Ghost
Artificial Analysis replaced its Intelligence Index today, October 27, 2026. Version 5 has nine evaluations, and AA says 60% of the Index is now private or held-out. Two evaluations are new, FrontierCode 2.0 and Terminal-Bench-Science; three were upgraded, HLE-Diamond, AA-Omniscience v1.1, and CritPt v2; three were removed, GDPval-AA, SciCode, and AA-LCR. AA also launched a Progress Index v1 alongside it.
Two things change for anyone who reads this site. First, every Artificial Analysis score we have printed since September, including the standings on our own Index page, is on the v4.3.2 scale, and v5 scores are not comparable to them. Second, the open-weight story changes shape. On v4.3.2 the best open model sat about 3 points behind the closed leader. On v5, by AA's own framing, that gap is about 12: the best released open weights score 32 against a closed leader at 58. The reason to read the methodology before the leaderboard is in the weights: FrontierCode 2.0 is a private benchmark built by Cognition, the company that sells Devin, and it alone is 15% of the score. We read the launch article, the evaluation page, and the methodology page. Here is what held up.
The v5 Board, as Published
| Model (effort) | Intelligence Index v5 |
|---|---|
| Claude Opus 5.5 (max) | 58 |
| GPT-6 Astra (max) | 57 |
| GPT-6.1 Sol (max) | 55 |
| Gemini 4 Argon (high) | 50 |
| Kimi K3.1 Pro (weights not released) | 46 |
| Claude Haiku 5.5 (max) | 36 |
| DeepSeek V4.1 Flash 1017 (max) | 32 |
| GLM-5.3 (max) | 32 |
Cost to run the Index, among the top ten AA shows: cheapest is GPT-6.1 Terra (max) at $0.96 per task; priciest is Claude Fable 5.1 (max) at $13.04.
Do not put a v5 number next to a v4.3.2 number without saying which is which. DeepSeek V4.1 Flash 1017 was 45 and a tie for second of 121 open models last week. On v5 it is 32. Nothing about the model changed; the ruler did. The same applies to Kimi K3.1 Pro, 55 on v4.3.2 and 46 on v5, and to every open-weight rank line we have printed this month.
Kimi's 46 is a closed API score. The weights are promised for November 4. Until they ship, the best released open weights on v5 are DeepSeek Flash 1017 and GLM-5.3 at 32, and both sit under Claude Haiku 5.5.
What Is in the Score, and Who Owns It
| Evaluation | Status in v5 |
|---|---|
| FrontierCode 2.0 | New; private; built by Cognition; 15% of the Index |
| Terminal-Bench-Science | New |
| HLE-Diamond | Upgraded |
| AA-Omniscience v1.1 | Upgraded |
| CritPt v2 | Upgraded |
| GDPval-AA, SciCode, AA-LCR | Removed |
| Share private or held-out | 60% (AA) |
Three honest observations:
- Fifteen percent of the frontier ranking is a test nobody outside Cognition can read. We raised the same catch when AA's Coding Agent Index v2.0 adopted a private Cognition evaluation. A private test resists contamination; it also resists audit, and the vendor that built it sells a coding agent scored on the same board.
- Held-out is a design choice with a cost. AA's case for 60% private is that public benchmarks leak into training data. The cost is that a lab disputing its score has nothing to re-run.
- The removals matter as much as the additions. GDPval-AA, SciCode, and AA-LCR were part of why some models scored where they did on v4.3.2. Their absence is part of why the open-weight gap widened; which part, AA's article does not break out.
Verified vs Unconfirmed: The Scorecard
| Claim | Verdict |
|---|---|
| Index v5 launched Oct 27; nine evaluations; 60% private or held-out | Verified (AA launch article) |
| FrontierCode 2.0 new, private, Cognition-built, 15% weight; Terminal-Bench-Science new; three upgrades; three removals | Verified (AA) |
| Opus 5.5 (max) 58; Astra (max) 57; Sol (max) 55; Argon (high) 50 | Verified (AA) |
| Kimi K3.1 Pro 46, weights not released; DeepSeek Flash 1017 (max) and GLM-5.3 (max) 32; Haiku 5.5 (max) 36 | Verified (AA) |
| Terra (max) $0.96 per task; Fable 5.1 (max) $13.04, among the top ten shown | Verified (AA) |
| Progress Index v1 launched | Verified (AA) |
| Open-weight gap about 3 points on v4.3.2 to about 12 on v5 | AA's framing of its own two boards |
| v5 rows for MiniMax M3.1 Flash, Muse Spark Open, Reflection Beam | Unverified (not in the launch article; check the live table) |
| Median output tokens per Index run on v5 | Unverified (not confirmed on the article body; omitted) |
What To Do
Re-cite every AA figure against the live v5 table before you compare models, and label the scale when you quote an older number. Our posts from before today carry v4.3.2 scores and will say so where it matters.
Read FrontierCode as a private Cognition test when you weigh the coding component, the same way you would read any vendor-built evaluation.
The frontier is elsewhere: on v5 the closed leaders are 55 to 58 and released open weights are 32, which is a different story from the one v4.3.2 told. Our model comparison has the current standings side by side, and our Index page will carry its read date and scale until we re-pull it on v5.
Update, October 29, 2026
- Oct 28: Google's Gemini 4 Carbon scored 59 (high) on v5, one point above Opus 5.5, and took the top row. Access is limited to Fairwind cyber defenders and trusted partners.
- Oct 29: Mistral Large 4 appears on v5 at 26 (high) and 23 (medium), per AA's model page, read for our weights post. The v5 rows for MiniMax M3.1 Flash, Muse Spark Open, and Reflection Beam are still unchecked here.
Sources
- Artificial Analysis: Intelligence Index v5
- Artificial Analysis: Intelligence Index evaluation page
- Artificial Analysis: intelligence benchmarking methodology
*Not sure which scale the model you are evaluating was scored on? Take the free 2-minute quiz and get matched. Then read the earlier catch on the same private benchmark: AA's Coding Agent Index v2.0.*