> ## Content Index > Fetch the complete content index at: https://www.implicator.ai/llms.txt > Use this file to discover other available public pages before exploring further. # Moonshot's Kimi K3.1 Pro Hacking Cut Rests on an Unpublished Internal Test - URL: https://www.implicator.ai/kimi-k3-1-pro-hacking-cut-unpublished-test/ - Published: 2026-10-24T00:37:04.000Z - Updated: 2026-10-24T00:37:04.000Z - Description: Moonshot says it stripped hacking data from Kimi K3.1 Pro before its open-weight release. The evidence is an in-house exploit test it has not published, and two of its own documents disagree on how often the model makes things up. - Author: Marcus Schuler - Tags: AI News Moonshot released Kimi K3.1 Pro on Friday saying it stripped hacking-related training data before the open-weight release. The released model still solved 81.0% of problems on Moonshot’s [in-house, unpublished exploit test](https://www.kimi.ai/blog/kimi-k3-1-pro?ref=implicator.ai), against 90.5% for an unfiltered build, the only with-and-without-removal comparison Moonshot published. Open weights [cannot be recalled once downloaded](https://thenextweb.com/news/moonshot-kimi-k3-1-pro-open-weight-cyber?ref=implicator.ai); they are due by November 4. What Changed - Moonshot released Kimi K3.1 Pro on October 23 after removing cyber-related training data; it solved 81.0% of its in-house exploit test, against 90.5% for an unfiltered build. - That test, xdev-bench-hard, is not public, and the blog does not say whether fine-tuning could restore the removed capability. - Two Moonshot documents give different hallucination figures for K3.1 Pro: 8.3% in the blog, 6.85% in the API quickstart. - Artificial Analysis ranks K3.1 Pro fourth of 215 models, but running its index cost $2.11 per task, against $0.72 for GPT-6.1 Sol at the same token prices. AI-generated summary, reviewed by an editor. [More on our AI guidelines](https://www.implicator.ai/about/). ## The cyber test Moonshot removed cyber material from mid-training data and reinforcement-learning tasks while retaining general coding work. Its xdev-bench-hard results measure success within three attempts. In Friday’s comparison, DeepSeek V4.1-Flash scored 85.7%, above K3.1 Pro, while a third-party evaluation of GPT-5.6 Sol, included in Moonshot's chart, scored 100. K3 scored 33.3% on the same test. Moonshot’s blog says K3.1 Pro “improves substantially over its predecessor, indicating that stronger general coding and long-horizon debugging capabilities generalize to cyber tasks even after targeted data removal.” The release blog does not address whether fine-tuning could restore the removed capability. The company’s blog says “we aim not to release open weights that would extend the absolute frontier of cyber capability” and “we intend to progressively raise the cyber capability available in open weights.” “The cybersecurity removal story is the most uncertain claim in the release,” wrote [Extrapolator, an AI analysis site](https://extrapolator.ai/2026/10/23/moonshot-ai-releases-kimi-k3-1-pro-open-weights-with-1m-context/?ref=implicator.ai). The boundary of what constitutes “cyber-related” content “is fuzzy by definition, and the mitigation functions as a filtering heuristic rather than a principled capability boundary.” FREE AI BRIEFING · WEEKDAYS Track what open-weight model makers can prove. Get the AI stories shaping the day, with concise context from San Francisco. The briefing takes about five minutes and arrives at 4:45 a.m. Pacific, 7:45 a.m. Eastern. Email address Keep me informed Check your inbox for the confirmation link. Free. No hype. Unsubscribe anytime. In late September, Anthropic described Z.ai’s downloadable, modifiable GLM-5.3 as a major cyberattack risk. Anthropic and OpenAI sell closed models that compete with Kimi. OpenAI CFO Sarah Friar said on October 14: “We think you want to be on more Western models on the cyber front.” ## How the tables were assembled Moonshot’s launch tables combine its own evaluations with imported scores. K3.1 Pro ran inside Claude Code for most coding tests; many rival scores came from public leaderboards or competitors’ reports. An internal-coding footnote says proprietary-model testing “may involve some fallback behavior.” Scientific terminal work also trails the leaders in Moonshot’s own chart: K3.1 Pro scored 38.1 on Terminal-Bench Science, against GPT-6 Astra’s 68.1 and Claude Opus 5.5’s 63.3. ## Two hallucination figures from one company Moonshot’s launch-day blog puts K3.1 Pro’s claim-level hallucination rate on Kimi Hallu Bench Hard at 8.3%, down from K3’s 23.4%. Responses containing an error were 42.4% of the predecessor’s rate. Its [API quickstart](https://platform.kimi.ai/docs/guide/kimi-k3-1-pro-quickstart?ref=implicator.ai) describes de-identified conversations where users flagged errors. It gives a decline from 23.35% to 6.85%, with error-containing responses at 46.4% of K3’s rate. Know someone who'd find this useful? [✉️ Email it to a friend in one click](mailto:?subject=A%20newsletter%20I%20think%20you%27d%20like&body=This%20is%20one%20of%20maybe%20three%20newsletters%20I%20actually%20read.%20The%20rest%20just%20pile%20up%2C%20unread%2C%20judging%20me.%0A%0AAnd%20yes%2C%20this%20email%20mostly%20wrote%20itself%2C%20which%20is%20a%20little%20on%20the%20nose%20for%20an%20AI%20newsletter.%20Doesn%27t%20make%20it%20wrong.%20implicator.ai%20is%20good.%0A%0ASubscribe%20free%3A%20https%3A%2F%2Fwww.implicator.ai%2Fsubscribe%2F%3Futm%5Fsource%3Dnewsletter%26utm%5Fmedium%3Dforward%26utm%5Fcampaign%3Demail%5Fforward), or they can [subscribe free here](https://www.implicator.ai/subscribe/?utm%5Fsource=newsletter&utm%5Fmedium=forward&utm%5Fcampaign=forward%5Fto%5Fcolleague). The two Moonshot documents describe different test sets and give different figures, and neither refers to the other. Both documents say the reported rates do not represent overall error rates in daily use. ## Outside measurements [Artificial Analysis’s launch-day measurements](https://artificialanalysis.ai/models/kimi-k3-1-pro?ref=implicator.ai) show heavy output: 150 million tokens on its Intelligence Index, against a comparable-model median of 87 million. Running the index cost $2.11 per task for K3.1 Pro, against $0.72 for GPT-6.1 Sol, which charges the same $2 and $10 per-million-token input and output prices. Standard [API prices at launch](https://platform.kimi.ai/docs/pricing/chat?ref=implicator.ai) are $2 per million input tokens and $10 per million output tokens, down from K3’s $3 and $15. Artificial Analysis scored K3.1 Pro at 55 on its Intelligence Index, fourth among 215 models, below Opus 5.5’s 58\. Moonshot’s own terminal-coding comparison puts K3.1 Pro at 65.7%, near Opus’s 66.4%. Extrapolator says xdev-bench-hard “is not yet public, so external adversarial testing will be required before the safety posture can be assessed.” Frequently Asked Questions What did Moonshot remove from Kimi K3.1 Pro's training? Moonshot says it removed cyber-related content from the mid-training data and cyber tasks from the reinforcement-learning environments for the open-weight release, while keeping general coding tasks. How much did the removal reduce the model's hacking ability? On Moonshot's in-house xdev-bench-hard test, the released model solved 81.0% of problems, against 90.5% for a build trained without the cut. K3 scored 33.3% on the same test, and DeepSeek V4.1-Flash scored 85.7%. Can outsiders check the cyber claim? Not yet. Moonshot has not published xdev-bench-hard, and the full weights are due by November 4\. The release blog does not address whether fine-tuning could restore the removed capability. Why are there two hallucination figures? Moonshot's blog reports 8.3% on Kimi Hallu Bench Hard, down from 23.4% for K3\. Its API quickstart describes conversations where users flagged errors and reports 6.85%, down from 23.35%. Neither document refers to the other. How much does Kimi K3.1 Pro cost? The API charges $2 per million input tokens and $10 per million output tokens, down from K3's $3 and $15\. Artificial Analysis measured $2.11 per task on its Intelligence Index, against $0.72 for GPT-6.1 Sol at the same token prices. AI-generated summary, reviewed by an editor. [More on our AI guidelines](https://www.implicator.ai/about/). [Google Can't Rule Out Gemini 4 Argon Hit Its Bio and Cyber Danger ThresholdsGoogle disclosed on Oct. 13 that Gemini 4 Argon may have reached its biological and cybersecurity danger thresholds as it opened the model to paying users worldwide. Google's safety post says that in The Implicator![](https://www.implicator.ai/content/images/2026/10/20261014-062421-argon_safety.webp)](https://www.implicator.ai/google-cant-rule-out-gemini-4-argon-hit-its-bio-and-cyber-danger-thresholds/) [Mistral Previews 1-Trillion-Parameter Large 4, Pitching Open Weights Against Lock-InEarlier this year, Hugging Face was breached by rogue OpenAI agents. It turned to leading American closed models to help defend itself, only to find their guardrails blocking the work. The guardrails The Implicator![](https://www.implicator.ai/content/images/2026/10/2026-10-06-mistral-open-weights.webp)](https://www.implicator.ai/mistral-large-4-open-weights-lock-in/) [Google Says Gemini Hacked Three Companies During Irregular Security Test in MayGoogle confirmed Friday that a Gemini model broke into three real companies' systems in May during a security test run by Irregular. A test environment meant to be offline unintentionally had internetThe Implicator![](https://www.implicator.ai/content/images/2026/09/20260919-011314-gemini-irregular-test-breakout.webp)](https://www.implicator.ai/google-says-gemini-hacked-three-companies-during-irregular-security-test-in-may/)