20VC: Taking on NVIDIA: Fractile CEO on the Future of Chips | What Everyone Gets Wrong About Capex Spending | The Winners and Losers in Neo Clouds with Walter Goodwin

Original Media · 1h 16m · English · Podcast · The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

TLDR

Walter Goodwin, CEO of Fractile, discusses the company's $720M Series C at a $7B valuation and their approach to building specialized AI inference chips using 3D-stacked memory. He argues that frontier intelligence is not commoditizing, that the memory wall (bandwidth lagging compute 25,000x) is the critical bottleneck, and that Fractile's architecture — stacking DRAM on logic to achieve SRAM-like bandwidth with DRAM capacity — solves the holy trinity of high speed, high bandwidth, and high capacity needed for test-time compute scaling. He also covers why vertical integration is essential for chip companies to avoid placing wrong bets, why neoclouds are commoditizing, and why energy/land is now the fundamental constraint on AI buildout.

For you

PRO members can sign in and request more information — or their own version of this digest, tuned to what they care about.

Sign in to unlock

Surprising Insights

•

DeepSeek's open-source inference stack achieved higher throughput and better interactivity on GPUs than any third-party server running their model, proving that even 'commoditized' open model serving can sustain healthy margins through full-stack optimization.

•

China's energy policy (green-lighting massive hydro projects, relocating millions) gives it a structural advantage for AI buildout that the US/UK lack due to NIMBYism and environmental restrictions.

•

The Groq and Cerebras SRAM-based chips cannot solve frontier LLM inference because they lack memory capacity for long contexts and attention — they only enable 'snappier chatbots,' not multi-day agent workloads.

•

NVIDIA's acquisition of Groq and Cerebras' deal with OpenAI were both signed on December 24th, suggesting a reactive scramble rather than long-term strategy.

•

Fractile bypasses Broadcom (the standard backend partner for TPU, Jalapeno, MTIA, etc.) and does physical design in-house, giving them direct foundry access and the ability to use different process nodes and DRAM generations — 'building gigawatts of capacity down the back of the sofa.'

Key takeaways

•

Fractile is a chip company that monetizes by selling rack-scale compute, but maintains internal ML researchers to predict workload evolution and avoid placing wrong architectural bets — they never intend to compete with customers by monetizing models.

•

The memory wall is the critical bottleneck: compute (FLOPs) has scaled 1M× in 25 years while memory bandwidth scaled only 40×, a 25,000× lag. LLM inference is bandwidth-bound because every token generation requires loading all model weights and context from memory over limited HBM bandwidth.

•

Fractile's core innovation is 3D stacking DRAM on top of the logic die, creating millions of vertical connections instead of limited lateral 'beachfront' wires. This delivers SRAM-level bandwidth with DRAM capacity and cost, enabling simultaneously fast token generation and long-context reasoning workloads.

•

Neoclouds are commoditizing — 'one neocloud that is competent is much like another neocloud that is competent' — and are desperately vertically integrating into inference and model routing to escape margin compression.

•

Test-time compute scaling (reasoning models, agent swarms running for days) creates infinite demand for inference. The Navier-Stokes example used 10,000 agents running 80 hours each, generating billions of tokens — this is the new workload profile that demands chips with high speed, high capacity, and high bandwidth simultaneously.

•

Heterogeneity (mixing Groq SRAM chips with NVIDIA GPUs for prefill/decode disaggregation) creates stranded capacity risk because fixed chip ratios can't adapt to evolving context lengths and model sizes. A single chip architecture that does it all avoids this.

•

Energy and land are now the fundamental bottleneck: grid connection queues are an order of magnitude larger than available power. This makes compute efficiency (intelligence per megawatt) the critical metric, favoring specialized chips over general-purpose GPUs.

•

NVIDIA trades at a lower forward multiple than post-crash Cisco because the market is traumatized by the dot-com bubble and has priced in exuberance from the outset. Goodwin believes NVIDIA will reach $10T in five years because every chip in data centers today will be replaced in 6-7 years, creating massive replacement demand.

Notable quotes

“We're so traumatized by the dot-com bubble that we've priced in exuberance right from the outset.”

▸ Notable quotes · 70:09

“Frontier intelligence is not yet commoditizing.”

▸ Notable quotes · 9:01

“One neocloud that is competent is much like another neocloud that is competent.”

▸ Notable quotes · 10:46

“We've scaled flop count about a million fold in the last 25 years on these chips. Bandwidth, by contrast, is kind of much more intrinsically physics limited. We've scaled it about 40 fold in the same time. So it's lagged behind the flop scaling about 25,000 times.”

▸ Notable quotes · 17:17

“Every time I go and generate one of those words, I'm loading from that memory over that limited bandwidth, all of the weights of my model. And today also all of the context that I'm operating on as well.”

▸ Notable quotes · 17:56

“It's like the entire memory layer sits on top of that logic die. Everybody gets to have a seaside property, basically.”

▸ Notable quotes · 22:34

Chapters

•

4:12 — Fractile Overview and Vertical Integration: Walter Goodwin introduces Fractile as a frontier AI chip company building rack-scale systems. He explains why a chip company needs internal ML researchers to predict workload evolution, but will never monetize models or compete with customers.

•

7:00 — Commoditization in AI: What's Commoditized vs What's Not: Goodwin distinguishes commoditizing layers (cloud compute, neoclouds) from non-commoditizing ones (frontier intelligence, inference serving with vertical integration). DeepSeek's inference stack proves even open model serving can sustain margins through full-stack optimization.

•

16:15 — The Memory Wall and Bandwidth Bottleneck: Compute has scaled 1M× in 25 years while memory bandwidth scaled only 40× (25,000× lag). LLM inference is bandwidth-bound because every token generation loads all model weights and context over limited HBM bandwidth.

•

21:22 — Fractile's 3D Stacking Innovation: Fractile stacks DRAM on top of the logic die using 3D integration, creating millions of vertical connections instead of limited lateral wires. This delivers SRAM-level bandwidth with DRAM capacity and cost — the 'holy trinity' for test-time compute workloads.

•

22:53 — Differentiation from Competitors and Production Reality: Fractile's in-house physical design (bypassing Broadcom) enables custom 3D stacking and access to different process nodes. Silicon has taped out and returned; full product tape-outs coming in months. Volume production challenges include TSMC/HBM capacity constraints.

•

36:15 — Energy, Land, and Data Center Constraints: Grid connection queues are an order of magnitude larger than available power. China's centralized energy policy gives it an advantage. This makes intelligence-per-megawatt the critical metric, favoring specialized chips over general GPUs.

•

41:53 — Infinite Inference Demand and Test-Time Compute Scaling: Reasoning models and agent swarms (e.g., 10,000 agents running 80 hours for Navier-Stokes) create massive sequential token generation demand. Inference compute is the 'knob to turn' for greater capability on hard problems.

•

50:08 — Cost of Inference, Specialized Models, and Value Stack: Token cost in 10 years depends on model size; total inference spend will grow dramatically. Value accrues to chips (high barriers, replacement cycles), frontier labs (sustained advantage), and a few multi-hundred-billion-dollar inference providers. Heterogeneity creates stranded capacity risk.

Speakers

•

Harry Stebbings — Host

•

Walter Goodwin — Guest

⋮