Gemini 4 Carbon: Google Reclaims the Top Spot with a Two-Week Update, But You Still Can’t Use It ...

Original Media · 11m · 中文 · YouTube · Best Partners TV

TLDR

Google released Gemini 4 Carbon, which tops the Artificial Analysis intelligence index at 59, edging out Claude Opus 5.5 and GPT-6 Astra. The model excels in cybersecurity (full vulnerability lifecycle) and long-horizon tasks (multi-day runs on 1M token context), with major reliability gains (cheating near zero, injection resistance 100%). However, it loses on several key benchmarks, its agent combo trails Claude Code, and it is not generally available — gated behind a U.S. government pre-release process and a trusted-partner program for defenders first. Google's Q3 revenue grew 25% but capex surged 114%, free cash flow turned negative, and the stock dropped ~10% after hours.

For you

PRO members can sign in and request more information — or their own version of this digest, tuned to what they care about.

Sign in to unlock

Surprising Insights

•

Despite being #1 overall, Carbon is beaten on several high-profile benchmarks (WebDev, FrontierSWE, OSWorld, coding agent combo, knowledge).

•

Hallucination rate nearly tripled (4%→11%) while correctness jumped (48%→58%) — a deliberate shift from 'cautious but silent' to 'answers more, mostly right'.

•

The 'world's best' model is unusable for most people; Google prioritizes giving it to defenders first due to dual-use risk.

•

Per-task cost is 23% lower than Opus 5.5 but still 24% higher than GPT-6 Astra.

•

Google's capex is accelerating (114% YoY, $195-205B full year, more in 2027) while free cash flow turns negative — the burn rate is outpacing revenue growth.

•

Two-week model cadence implies the #1 leaderboard shelf-life may be only two weeks.

Key takeaways

•

Gemini 4 Carbon scores 59 on Artificial Analysis v5.0, claiming the #1 spot from Claude Opus 5.5 (58) and GPT-6 Astra (57).

•

Cybersecurity is a standout: Carbon leads on vulnerability discovery (SEC-bench Pro 88.2%), exploitation validation (ExploitGym 54.8% vs 42.4%), and patching (CWE-bench 71%, 81% with 4 tries).

•

Long-horizon capability crosses a threshold: 1M token context, runs for days on large codebases, GraphWalks 96.2% (vs 71.8%/66.8%), Terminal-Bench 65.2%.

•

Reliability improved sharply: cheating 0.0%, unsupported assertions halved, privilege escalation -26%, prompt injection 0/15, high-severity alignment alerts -80%.

•

Trade-off: hallucination rose from 4% to 11% (still far below Opus/Astra ~50%) as the model answers more instead of refusing.

•

Notable weaknesses: WebDev Arena #4, FrontierSWE 2 behind Astra, OSWorld-2.1 behind Opus, coding agent index (with Antigravity) 59 vs 62 for Opus+Claude Code, knowledge tasks behind Grok/Muse/Kimi.

•

Release is staged: U.S. government pre-release testing → cybersecurity defenders via Fairwind program → gradual expansion; no public timeline.

•

Google Q3: revenue $127.6B (+25%), Cloud $30.7B (+103%, margin 36.2%), $561B backlog; but capex $51.4B (+114%), free cash flow -$9.1B, net income -24%, stock -10% after hours.

Notable quotes

“Carbon 比 Argon 强太多了,我真希望他们当初干脆跳过 Argon 直接发这一版”

▸ Notable quotes · 0:35

“谷歌现在在周期性地发布模型”

▸ Notable quotes · 0:24

“一个找漏洞、验证漏洞、补漏洞都做到第一的模型,本身就是一把双刃剑”

▸ Notable quotes · 6:43

“榜单发得震天响,普通用户点进去却发现用不了”

▸ Notable quotes · 7:00

“当一个模型的发布节奏快到两周一次,这说明榜单第一的保质期可能也就只有了两周”

▸ Notable quotes · 10:26

“收入和订单在翻倍,但是烧钱的速度翻倍得更快”

▸ Notable quotes · 9:42

Chapters

•

0:00 — Release & Benchmark Overview: Google launches Gemini 4 Carbon two weeks after Argon; Carbon tops Artificial Analysis index at 59, beating Opus 5.5 and GPT-6 Astra. CEO Pichai jokes about periodic elemental codenames.

•

1:16 — Cybersecurity Capabilities: Carbon leads full vulnerability lifecycle: discovery (SEC-bench Pro 88.2%), validation (ExploitGym 54.8%), patching (CWE-bench 71%, 81% with retries). Internal and partner data show 2x+ security code gen vs Argon, 10x+ patch quality vs competitor, Wiz confirms penetration testing lead.

•

2:58 — Long-Horizon Tasks: 1M token context enables multi-day runs on large codebases. GraphWalks 96.2% (vs 71.8%/66.8%), Terminal-Bench 65.2% (vs 59.6%/59.1%). Model self-corrects from dead ends.

•

3:40 — Weaknesses & Limitations: Not universally #1: WebDev Arena #4, FrontierSWE 2 behind Astra, OSWorld-2.1 behind Opus, coding agent index (Antigravity) 59 vs 62 (Opus+Claude Code), knowledge tasks trail Grok/Muse/Kimi.

•

4:32 — Reliability & Alignment Improvements: Cheating 0.0%, shortcut attempts 0.4%, unsupported assertions halved, privilege escalation -26%, corner-cutting -29%, prompt injection 0/15 blocked, high-severity alerts -80%. Correctness up (48%→58%) but hallucination up (4%→11%), still far below rivals (~50%).

•

6:05 — Release Strategy & Availability: Staged rollout: U.S. government pre-release testing → cybersecurity defenders via Fairwind trusted partner program → gradual expansion. No public timeline. Dual-use rationale: give powerful vulnerability tool to defenders first. Community frustration over 'release but unavailable'.

•

7:10 — Business Context & Financials: Same-day earnings: revenue $127.6B (+25%), Cloud $30.7B (+103%, margin 36.2%), $561B backlog. API 32B tokens/min (+4x), 1B Gemini users, 90% Fortune 100 on Enterprise. Search $63.4B (+12%), AI Mode queries 3x longer. TPUs now sold externally (Oracle, CoreWeave).

•

9:40 — Industry Implications & Summary: Capex $51.4B (+114%), full-year $195-205B, 2027 higher. Free cash flow -$9.1B, net income -24%, stock -10% after hours. Revenue doubling but burn doubling faster. 2-week cadence suggests #1 shelf-life ~2 weeks. RSI speculation unconfirmed.

Speakers

•

大飞 (Dafei) — Host

⋮