> ## Content Index > Fetch the complete content index at: https://www.implicator.ai/llms.txt > Use this file to discover other available public pages before exploring further. # Artificial Analysis Raises the Bar for AI Models - URL: https://www.implicator.ai/artificial-analysis-index-v5-open-model-gap/ - Published: 2026-10-28T16:02:25.000Z - Updated: 2026-10-28T16:02:25.000Z - Description: Artificial Analysis changes the tests behind its AI rankings, with private data now carrying most of the weight. The wider gap for open models comes with a catch about which weights are available, while benchmark costs tell a different part of the story. - Author: Marcus Schuler - Tags: AI News, #home-companion Artificial Analysis [released a revised Intelligence Index](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5?ref=implicator.ai) on Tuesday. Models with downloadable weights trail the proprietary leader by 26 points on the revised suite. The results do not establish that individual models have deteriorated. Evaluations with private datasets or withheld answers now account for 60% of the index’s weighting. The independent benchmarking organization’s [new methodology](https://artificialanalysis.ai/methodology/intelligence-benchmarking?ref=implicator.ai) combines nine evaluations, with knowledge work, coding, science and general capabilities each receiving a quarter of the total weight. In that 60% of the index, the evaluations keep private test questions or held-out answers from developers before testing. What Changed - Private datasets or withheld answers now account for 60% of Artificial Analysis's revised Intelligence Index weighting. - Among models with released weights, DeepSeek V4.1 Flash 1017 and GLM-5.3 trail the overall leader by 26 points. - GPT-6.1 Sol costs $1.27 per benchmark task at maximum effort, against $10.04 for Claude Opus 5.5. - Changed tests and weights make index versions unsuitable for long-term comparison; a separate Progress Index addresses that purpose. AI-generated summary, reviewed by an editor. [More on our AI guidelines](https://www.implicator.ai/about/). ## Scores on the revised suite In Artificial Analysis’s October 27 release results, Claude Opus 5.5 leads with an index score of 58, followed by GPT-6 Astra at 57 and GPT-6.1 Sol at 55\. All three scores use maximum reasoning effort, the setting controlling how much work a model can spend on a response. Kimi K3.1 Pro scores 46, putting it 12 points behind the leader, compared with a three-point gap under version 4.3\. Its weights have not yet been released. Changes to tests and weights make Intelligence Index scores unsuitable for long-term comparisons across versions. Among models whose weights are actually available, DeepSeek V4.1 Flash 1017 and GLM-5.3 tie at 32 at maximum effort, leaving a 26-point gap to the leader on the revised index. Open weights means downloadable model parameters; it does not necessarily mean the accompanying software is open source. FREE · ABOUT FIVE MINUTES Follow the tests behind AI model rankings. Implicator cuts the flood to the developments worth your attention, with concise reporting on what changed and why it matters. Delivered every weekday at 4:45 a.m. Pacific, 7:45 a.m. Eastern. Email address Get the free briefing Check your inbox for the confirmation link. From San Francisco. No spam. Unsubscribe anytime. These are Artificial Analysis’s measurements. The index primarily tests English-language text tasks and does not measure suitability for every application. ## Coding changes must survive review The new FrontierCode 2.0 benchmark contains 132 private coding tasks drawn from real software repositories. It grades whether a maintainer would accept the change, and failing any blocking check earns zero. Artificial Analysis runs each task three times in its Stirrup testing system. In its October 27 results, Opus 5.5 at maximum effort scores 46% on FrontierCode 2.0. A [published walkthrough from Cognition](https://cognition.com/blog/frontier-code-2.0?ref=implicator.ai), the coding-agent company that created the benchmark, shows a failure by the overall index leader. Opus 5.5 fixes a reported bug in Mock Service Worker, a library used to simulate network requests, but discards the old generator without running its finally-block cleanup. The immediate bug disappears, while the skipped cleanup can leave resources unreleased or the application in an inconsistent state. In Cognition’s example, Opus fails the task at every reasoning setting. That walkthrough is the benchmark creator’s demonstration, separate from Artificial Analysis’s composite results. Know someone who'd find this useful? [✉️ Email it to a friend in one click](mailto:?subject=A%20newsletter%20I%20think%20you%27d%20like&body=This%20is%20one%20of%20maybe%20three%20newsletters%20I%20actually%20read.%20The%20rest%20just%20pile%20up%2C%20unread%2C%20judging%20me.%0A%0AAnd%20yes%2C%20this%20email%20mostly%20wrote%20itself%2C%20which%20is%20a%20little%20on%20the%20nose%20for%20an%20AI%20newsletter.%20Doesn%27t%20make%20it%20wrong.%20implicator.ai%20is%20good.%0A%0ASubscribe%20free%3A%20https%3A%2F%2Fwww.implicator.ai%2Fsubscribe%2F%3Futm%5Fsource%3Dnewsletter%26utm%5Fmedium%3Dforward%26utm%5Fcampaign%3Demail%5Fforward), or they can [subscribe free here](https://www.implicator.ai/subscribe/?utm%5Fsource=newsletter&utm%5Fmedium=forward&utm%5Fcampaign=forward%5Fto%5Fcolleague). The other addition, Terminal-Bench-Science, tests 70 expert-curated research workflows and requires every test for a task to pass. In Artificial Analysis’s release setup, Astra at maximum effort completes 63%, ahead of Opus at 59% and Sol at 58%. ## Cost and comparisons over time The October 27 cost figures show Sol at maximum effort costing $1.27 per Intelligence Index task, against $10.04 for Opus at maximum effort. Sol uses about one-eighth the spending per Intelligence Index task, with three fewer index points. Those prices measure spending on benchmark tasks, rather than the cost of successfully completing a business assignment. The revised suite also replaces Humanity’s Last Exam with HLE-Diamond and updates its knowledge and physics evaluations. HLE-Diamond contains 1,000 questions, split evenly between reasoning and knowledge. Artificial Analysis launched a separate [Progress Index](https://artificialanalysis.ai/methodology/progress-index?ref=implicator.ai) alongside the revision. It estimates model capability and task difficulty together, linking models through tests they share so that systems released years apart can be compared on one continuous scale. Frequently Asked Questions What does Intelligence Index v5 measure? It combines nine evaluations, giving equal weight to knowledge work, coding, science and general capabilities. The suite primarily tests English-language text tasks. Which models lead the revised index? The October 27 release results put Claude Opus 5.5 at 58, GPT-6 Astra at 57 and GPT-6.1 Sol at 55, all at maximum reasoning effort. Which leading models have downloadable weights? DeepSeek V4.1 Flash 1017 and GLM-5.3 tie at 32 at maximum effort. Kimi K3.1 Pro scores 46, but its weights had not been released. Does the wider gap mean open models got worse? The revised tests and weights changed the comparison. The wider measured gap does not establish that individual models deteriorated, and scores across versions are unsuitable for long-term comparisons. What do the reported task costs mean? At maximum effort, GPT-6.1 Sol costs $1.27 per Intelligence Index task and Claude Opus 5.5 costs $10.04\. These are benchmark-task costs, rather than the cost of successfully completing a business assignment. AI-generated summary, reviewed by an editor. [More on our AI guidelines](https://www.implicator.ai/about/). [Meta Releases Muse Spark Open, an 816B Model That Trails Chinese Rivals at CodingMeta released the weights for Muse Spark Open on Thursday, a model it calls the most capable U.S. open-weight model to date. On Terminal-Bench 2.1, a coding test in Meta's own table, it scores 76.4 agThe Implicator![](https://www.implicator.ai/content/images/2026/10/20261016-012254-muse_spark_macs.webp)](https://www.implicator.ai/meta-releases-muse-spark-open-an-816b-model-that-trails-chinese-rivals-at-coding/) [Mistral Previews 1-Trillion-Parameter Large 4, Pitching Open Weights Against Lock-InEarlier this year, Hugging Face was breached by rogue OpenAI agents. It turned to leading American closed models to help defend itself, only to find their guardrails blocking the work. The guardrails The Implicator![](https://www.implicator.ai/content/images/2026/10/2026-10-06-mistral-open-weights.webp)](https://www.implicator.ai/mistral-large-4-open-weights-lock-in/) [Z.ai Delays GLM-5.3 Weights Two Weeks After Cyber Score Beats Mythos 5Beijing-based Z.ai released GLM-5.3 on Friday while holding back the model's downloadable weights and gating its most sensitive cybersecurity functions. In launch tests, GLM-5.3 scored 84.5% on CyberGThe Implicator![](https://www.implicator.ai/content/images/2026/08/2026-08-15-03.27.10-z-ai-delays-glm-5-3-weights-cyber-score-mythos-5@2x.webp)](https://www.implicator.ai/z-ai-delays-glm-5-3-weights-two-weeks-after-cyber-score-beats-mythos-5/)