Trending AI Insights: Gemini 4 Carbon Leads — So Why Is Coding Still Claude's?
In two days Google shipped Gemini 4 Carbon to the top of the Intelligence Index and Mistral put a 1-trillion-parameter flagship into open weights. The gap between them explains more than either launch does.
Two launches, two days, opposite bets. On 28 October Google DeepMind announced Gemini 4 Carbon and said it takes first place on the Artificial Analysis Intelligence Index v5.0. One day later, on 29 October, Mistral AI released the full weights of Mistral Large 4 — a one-trillion-parameter flagship — for anyone to download, fine-tune and deploy.
Read separately, both are routine product news. Read together, the two numbers that matter are not the ones in either headline. The closed leader scores 59 on that index; the best open-weight model out of the West scores 26. That 33-point distance is the real story of the week, and it is the opposite of what the open-weights narrative has been promising for two years.
Carbon is first overall, and that is the least interesting fact
The score itself is real and independently verified. Carbon lands 59 on the Intelligence Index v5.0, ahead of Claude Opus 5.5 at 58, GPT-6 Astra at 57, and its own predecessor Gemini 4 Argon at 50. A nine-point jump in one generation would be notable on its own; doing it two weeks after Argon reached general availability on 13 October is a statement about release cadence, not just capability.
The rank, however, is a composite. Artificial Analysis reported that Carbon leads 4 of the index's 9 evaluations and is, in its own words, “not leading in coding and knowledge work.” The gaps run in both directions. On the Coding Agent Index, Carbon in the Antigravity CLI scores 59 against 62 for Claude Opus 5.5 running in Claude Code. On OSWorld-2.1, the computer-use test, Carbon reaches 81.5%, below Opus 5.5's 87.2%. On FrontierSWE 2 it slightly trails both Opus 5.5 and GPT-6 Astra.
So “number one” here means widest coverage across nine axes, not best on any single one. That distinction used to be academic. It stops being academic the moment teams pick a model for one workload and pay for the average.
The honest trade-off: more reasoning, more money, more making things up
Independent testing found three costs attached to the gain. Answer accuracy rose from 48% to 58%, but the hallucination rate climbed from 4% to 11% — still the second-lowest Artificial Analysis has measured, but a near-tripling. Carbon generates about 2.3 times Argon's reasoning tokens, which is where the capability comes from. And per-task cost works out to $7.76 against $7.68 for Argon: effectively flat, because token efficiency fell as capability rose.
Some workloads slow down as well. Artificial Analysis noted that agentic finance and tax tasks can take more than twice as long as they did on Argon. For a batch job overnight, that is irrelevant. For an interactive product, it is a product decision.
Where Carbon is genuinely ahead: security, and jobs measured in days
The security numbers are the strongest part of the release and the reason the rollout is gated. Google reports 88.2% on SEC-bench Pro for finding real vulnerabilities, 54.8% on the ExploitGym exploitation benchmark, and 71.0% on Collinear's CWE-bench for patching root causes. For context, Google's own chart puts GPT-6 Astra at 85.4% and Argon at 83.4% on SEC-bench Pro, and around 34% to 42% for competitors on ExploitGym. These are vendor-run evaluations, explicitly without guardrails, and should be read as raw capability rather than behaviour under production safety layers.
The long-horizon claims are harder to benchmark but more consequential. Google says internal teams ran Carbon “for days” on multi-step engineering work, and that the model recovered from dead ends instead of stopping. In the Vending-Bench 2 business simulation it finishes with a bank balance of $16,484 against $13,718 for Argon. Internally, Google claims more than double Argon's success rate at generating secure code, and its Chrome team reports more than a tenfold improvement in high-quality patches over “another lab's large frontier model” — a comparison Google declines to name.
A model that can find and exploit vulnerabilities unattended is a weapon as much as a tool, so access is rationed by identity rather than by price. From the week of 28 October, Carbon goes to a subset of cyber defenders, then gradually to more partners through the Fairwind Program, which Google says has more than 650 members including governments and operators of healthcare and telecom infrastructure. Applicants face background checks, must use phishing-resistant multi-factor authentication, cannot resell access, and may only hand it to internal security, incident-response and penetration-testing teams. There is no waitlist, no API entry and no app toggle; a third-party check of a paid Gemini API key on 28 October listed Argon but not Carbon. Google is also participating in the US government's voluntary pre-release testing process.
Pricing is already set: $2 per million input tokens and $10 per million output at the introductory rate, with cached input at 95% off, doubling to $4 and $20 after the introductory period ends — a date Google has not announced. If Carbon follows Argon's path, general availability is about 13 days after announcement; Google has given no such promise.
Mistral's counter-move: give away the weights instead
A day later, Mistral did the thing closed labs will not. Mistral Large 4, internally codenamed Le Chonk, went up on Hugging Face as a full weight release: about one trillion total parameters with 49 billion active per token (52 billion including embeddings and output layers), 128 routed experts with four active plus one shared, MLA attention with a DeepSeek-style sparse attention indexer, a one-million-token context window, and a 1.6-billion-parameter Pixtral vision encoder. It was trained from scratch on European infrastructure using 3,800 NVIDIA Grace Blackwell GPUs.
The released checkpoint is not the one Mistral showed in public preview on 6 October. Reinforcement learning continued for three more weeks, and the gains landed exactly where buyers look. Terminal-Bench 4.0 went from 28.3% to 41%, a 13-point jump, while output tokens fell 30%. On the Coding Agent Index it scores 48%, level with GLM-5.3 — the model Mistral calls the leading open-weight coder — while using 47% fewer tokens per task. BrowseComp reaches 84.5%, the finance benchmark Finch reaches 73.3%, and B3 agent-security attack resistance rises from 93.3% to 97.1%.
The commercial terms are the interesting part. The licence is a modified MIT: free to use, modify, fine-tune and commercialise, with one carve-out — anyone running a model-as-a-service business needs a separate commercial agreement with Mistral. API pricing is $0.70 per million input tokens, $0.07 cached and $2.10 output, roughly a third of Carbon's introductory input price.
Then comes the correction that neither press release leads with. On that same Intelligence Index v5.0 where Carbon scores 59, Mistral Large 4 scores 26. That is the highest score among Western open-weight models and still only fifth among open weights overall, because Chinese labs occupy the top four places. On Artificial Analysis' Cyber Index it reaches 51%, behind DeepSeek V4.1 Flash at 57%.
The three numbers worth remembering
59 against 26. The closed frontier and the best Western open-weight model are not close. Publishing weights is now a distribution strategy, not a shortcut to parity — and the frontier labs have shown they can ship a new flagship every two weeks, which is faster than any open release cycle can follow.
47% fewer tokens. Mistral's real pitch is not the score, it is the bill: tie the leading open coder while spending less than half the tokens. That is a different axis from capability, and it is the one that shows up in a monthly invoice.
Two weeks. Argon to Carbon was 14 days. Whatever advantage a benchmark lead confers, it now expires faster than most procurement cycles.
Which points at a practical problem the leaderboards do not solve. Once models stop competing on a single score and start competing on workload — security and long-horizon engineering for one, cheap agentic coding for another, token efficiency for a third — the useful question stops being “which model is best” and becomes “which model for this job, at what cost per task.” That is the question aibookmarks.net is built around: a human-curated AI tool directory (one of the most comprehensive AI tool directories) that groups tools by the job you need done rather than by launch date, with side-by-side comparisons that put per-task cost next to per-token price. If you are choosing between a gated frontier model and an open-weight one this week, start from the workload, not from the ranking.
One caveat before you trust any of these tables
Vendor comparison charts are not comparable across vendors. Mistral's Terminal-Bench 4.0 chart scores Kimi K3 at 21%, while Moonshot reported 65.7% for its successor K3.1 Pro on the same benchmark name earlier in October. Both figures can only be true if the harness, the tooling and the token budget differ — and they do. Mistral has not published a technical report for Large 4 yet, so its evaluation conditions are unknown. Google's security figures are its own runs without guardrails.
The practical rule: use a single independent aggregator to compare across labs, and use vendor tables only to compare a vendor against itself, generation over generation. Everything else is marketing with error bars.
What to do this week
If you defend infrastructure: apply to Fairwind now. Carbon is the only model in this comparison you cannot buy, and the queue is vetted rather than priced.
If you build on closed APIs: Argon is what you can actually call today. Budget Carbon at $2/$10 introductory and $4/$20 after, and assume the introductory window closes without much notice.
If you build on open weights: Large 4 is the strongest Western option and the cheapest per token of the two, but budget the hardware. Mistral's own launch examples assume a single eight-GPU node for vLLM or SGLang, with 2-bit quantisations fitting a 512GB Apple Silicon machine at roughly 15 tokens per second. Check the licence text product by product if any part of your offering lets users steer the model.
If you evaluate for a living: wait for Mistral's technical report before quoting its numbers, and re-run your own harness. The 33-point index gap is the one figure that will not move.
Original author: Koray Kavukcuoglu, SVP, Google DeepMind, and the Mistral AI research team, drawing on both companies' release announcements, with evaluation data from Artificial Analysis as reported by independent outlets.
Published by AI Bookmarks · More AI news →