TLDR
Jon Stewart interviews AI experts Paul Scharre and Sarah Shoker about military AI use, the Anthropic-OpenAI controversy over defense contracts, autonomous weapons, and governance challenges. They discuss how AI like Claude is integrated into targeting systems (Maven Smart System), the debate over human-in-the-loop policies, and the need for international norms and congressional oversight.
For you
PRO members can sign in and request more information — or their own version of this digest, tuned to what they care about.
Sign in to unlockSurprising Insights
AI models escalate more aggressively than humans in nuclear war-game simulations, and researchers don't fully know why.
The same consumer LLMs (Claude, ChatGPT) are plugged into classified military targeting systems with minimal modification.
Anthropic's refusal to bend on its red lines may trigger severe government retaliation beyond contract loss, including seizure of models.
AI systems can be fine-tuned with 'personalities' (e.g., more acquiescent), raising the risk of sycophantic advice in life-and-death decisions.
The Maven Smart System's chatbot personality is unknown even to experts — 'we don't know the personality of the weapon.'
Key takeaways
Military AI use is largely mundane (logistics, intelligence analysis) but increasingly includes targeting support via systems like Maven Smart System using Anthropic's Claude.
Anthropic and OpenAI have similar military-use policies (no autonomous weapons, no mass surveillance); their public feud reflects personality clashes more than substantive differences.
AI enabled 1,000 targets in Iran in one day — double the 2003 Shock and Awe pace — raising concerns about speed outpacing human judgment.
Large language models show a tendency to escalate more aggressively than humans in war-game simulations, possibly due to training data biases.
Governance is stuck: UN consensus bodies are ineffective; experts favor hardware-level controls (advanced chips) as a choke point for guardrails.
The $200M defense contract is trivial for AI companies; the real risk is government retaliation (e.g., Defense Production Act) against firms that resist military demands.
Congress has tools (hearings, funding, classified briefings) but lacks political will; public pressure and all-of-society cooperation are essential.
Quantum computing is not an imminent game-changer for AI; exponential growth remains in classical AI chips, not quantum.
Notable quotes
“Conceptually, the idea would be, who's choosing the targets? If a human chooses the targets, then you'd say the human is in the loop. If the AI is choosing it, or the AI is recommending that humans aren't really paying any attention, then you'd say, well, the machine is doing that.”
▸ Notable quotes · 0:00“It is incredibly chilling. And just recently, a huge controversy broke out into the open when one AI company, Anthropic, drew a line and said, we shall not allow our product to be used in this way. And then another AI company there, OpenAI went, we will. That's cool with us.”
▸ Notable quotes · 1:27“The first day that the production of 1,000 targets in Iran has largely been credited to the MSS, the Maven Smart System.”
▸ Notable quotes · 15:00“We know that Anthropic's AI, Claude, is deployed on US military classified networks. It's integrated through the Maven smart system... and it's been used by the US military in real-world operations, including the operation against Venezuelan president Maduro and operations in Iran.”
▸ Notable quotes · 20:34“I really would resist any kind of narrative that tries to identify a hero and a villain in this story. I'm not necessarily sure that those are appropriate roles for either Anthropic or OpenAI.”
▸ Notable quotes · 32:42“The question is, who gets to set the rules? And the Pentagon's answer is, we get to set the rules. We don't want these companies dictating to us.”
▸ Notable quotes · 36:50“The models have a tendency to escalate more aggressively than humans would. And it's not really clear why the models do that.”
▸ Notable quotes · 49:40“One possibility going forward is you could see a world where as more and more military power gets embedded into software and data and AI, it's kind of hard to measure that. You might see more miscalculation.”
▸ Notable quotes · 51:57“It makes us feel better because it's, therefore, objective, and it removes the human qualitative or subjective element to it. But these models are not always going to be reliable because they are, in fact, statistical prediction machines.”
▸ Notable quotes · 53:55“The best avenue we have is starting at the level of AI hardware and then building guardrails domestically, eventually globally, from the ground up. These chips are made in one place on Earth: Taiwan.”
▸ Notable quotes · 63:51“What would've been a private conversation is now happening publicly. Kind of messy. A lot of personalities involved on all sides. But it's airing this issue, and then we're all sort of debating, well, what should be these red lines? That's a good conversation to have.”
▸ Notable quotes · 74:48Chapters
0:00 — Introduction: AI and the Military: Jon Stewart introduces the episode on AI in military applications, the Anthropic-OpenAI controversy, and the chilling implications of LLMs in targeting.
3:00 — Guest Backgrounds: Paul Scharre (CNAS, former Army Ranger, Pentagon autonomous weapons policy lead) and Sarah Shoker (UC Berkeley, former OpenAI geopolitics lead) outline their expertise.
4:20 — How the Military Uses AI: AI as an optimizer for logistics, intelligence analysis (computer vision), and increasingly large language models for decision support — not just consumer-style chat.
14:00 — Maven Smart System and Claude: Details on Palantir's Maven system integrating Anthropic's Claude for target generation and prioritization in Iran and Venezuela; 1,000 targets in one day.
20:00 — The Anthropic vs. OpenAI Controversy: Both companies have similar red lines (no autonomous weapons, no mass surveillance); feud stems from contract negotiations and personality clashes, not policy gaps.
36:40 — Governance and Congressional Role: Debate over who sets rules: companies, Pentagon, or Congress. Congress has oversight tools but lacks political will; international UN process stalled by consensus requirement.
49:30 — Risks: Escalation, Sycophancy, Speed: AI war-game escalation bias, sycophantic model behavior, and the danger of faster targeting outpacing human accountability and critical thinking.
64:00 — Hardware Choke Points and Quantum: Advanced chip supply chain (Taiwan, US, Japan, Netherlands) as a regulatory lever; quantum computing not an imminent disruptor for AI.
74:00 — Wrap-up and Post-Show Discussion: Experts emphasize public debate as silver lining; production team discusses pizza, habits, and the surreal nature of AI personalities in weapons.
Speakers
Jon Stewart — Host
Paul Scharre — Guest Expert
Sarah Shoker — Guest Expert
Gillian Spear — Producer
Lauren Walker — Lead Producer
Brittany Mehmedovic — Producer