AWS 108-Page Report: Why AI Isn't Delivering Value | AI Implementation | AWS | Agentic AI | AI RO...

Original Media · 23m · 中文 · YouTube · Best Partners TV

TLDR

An AWS Executives in Residence team interviewed 154 leaders across 23 industries over nine months to uncover why most enterprise AI investments fail to deliver value. The study identifies ten structural breakpoints — from confusing efficiency with value, to measurement traps, adoption plateaus, skills gaps, governance mismatch, cost opacity, data layer neglect, platform design, outdated work processes, and leadership identity crises — that stall AI at the pilot stage. Organizations that cross these breakpoints share one trait: they continuously shorten the distance between knowing and doing, treating every solution as a hypothesis with an expiry date.

For you

PRO members can sign in and request more information — or their own version of this digest, tuned to what they care about.

Sign in to unlock

Surprising Insights

•

Early cost transparency correlates with higher AI spend, not lower — because visible ROI justifies bigger bets.

•

Only <5% of advanced AI users cited "leadership encouragement" as helpful; peer credibility drives adoption, not top-down permission.

•

The most committed-looking teams (detailed roadmaps, defended architectures) are often the slowest — their rigor masks "continuing what we already decided" instead of testing the next hypothesis.

•

Data quality improves by fixing it *during* application development, not by waiting for a clean-data program — companies that wait never start.

•

Shadow AI at 10x the scale of shadow IT isn't a governance failure; it's a signal that governance was designed to obstruct work, not enable it.

Key takeaways

•

Efficiency, productivity, and value are three distinct stages; most AI projects stop at efficiency (doing the same work faster) without reinvesting freed capacity into a named growth initiative within the same quarter, so the time savings get absorbed by more low-value work.

•

Adoption rate is a trap: one company had 88% adoption but fewer than 1 in 5,000 sessions produced genuinely better work. The report recommends pairing every efficiency metric with a quality counter-metric (e.g., tickets closed per day paired with customer satisfaction) and running both for 90 days before trusting either.

•

Adoption maturity has five levels; the critical inflection is reaching level 2 (handling discrete tasks). Below level 2, only 54% find AI useful; above level 3, it exceeds 85%. Top-down "go ahead and experiment" emails don't work; peer-to-peer teaching, leaders visibly using AI, and progressively harder challenges drive real adoption.

•

The talent pipeline is breaking: judgment used to be forged through repetitive execution that AI now automates. No organization has a proven answer for how juniors will develop judgment. Practical mechanisms include quarterly "capabilities we deliberately abandoned" reviews, cross-domain rotations, and changing promotion criteria to reward unlearning and relearning, not classroom training.

•

Governance designed for six-month projects strangles six-day AI experiments, pushing usage underground (shadow AI). Effective governance lets agents "earn" autonomy through a probation period, places guardrails outside the model at the system layer, and tiers controls by consequence severity — e.g., a decision tree asking: personal data? real-time? human-in-the-loop? — with fast lanes for low-risk use cases.

•

AI costs have no historical baseline and spike with each new capability wave. Making token spend visible by team and use case — not to spend less, but to prove what each dollar returns — lets companies double down confidently. AT&T tracks 270B tokens/day and requires every dollar to yield >$5 free cash flow; costs must sit on the P&L of the team that defines the need and captures the value.

•

Enterprise data has two layers: structured systems (CRM, databases) and unstructured human context (emails, chats, meeting notes). The second layer holds the undocumented expertise — why decisions were made, how workarounds emerged. Data quality isn't achieved by cleaning upfront; it's measured by how often humans correct AI outputs in production, and improved by fixing data incidentally while solving real problems.

•

A production AI platform needs four components: orchestration, evaluation/observability, a model gateway, and a context layer. The context layer is more valuable than the model; the tenth agent costs a fraction of the first because it reuses the same governance gate. Context freshness must be actively governed — stale documents, not model drift, cause accuracy decay.

•

Work processes are still designed for a world where execution is expensive: approvals, reviews, and handoffs consume 91% of project time even when execution takes one day. Organizations must codify repeated judgments into reusable assets, write org context in machine-readable formats, and treat every redesign as a hypothesis, not a commitment — building the capacity to discard fast matters more than building right the first time.

•

Leadership is shifting from manager to builder. CEOs are becoming de facto Chief AI Officers, doing three things: making experiments small so failure is cheap, publicly praising early problem-spotters, and accepting that their own evaluation standards for AI output will keep changing. The ultimate competitive asset is learning rate, not any particular solution.

Notable quotes

“如果我一小时干完了四小时的活,但是没有任何安排,告诉那省下来的三小时该去哪儿,那公司什么也没得到”

▸ Notable quotes · 1:44

“你的 AI 投资是生产'昨天的更快版本',还是在资助客户以前拿不到的新东西?”

▸ Notable quotes · 2:03

“用使用量来衡量 AI,最终结果就是你只知道谁登录了,不知道谁长出了把使用变成价值的能力”

▸ Notable quotes · 3:23

“过去几十年,多数企业的人才培养体系是围绕着'执行能力'构建的,而执行,恰恰是 AI 现在最擅长的事”

▸ Notable quotes · 7:23

“治理不是成本项,是利润差”

▸ Notable quotes · 10:58

“超出 token 预算是一个'高质量的问题',因为这说明大家真的在用系统”

▸ Notable quotes · 13:03

Chapters

•

0:00 — The AI Value Gap: 84% of executives believe agentic AI will transform their business, yet only 6% deploy it in core processes; 79% plan to increase spend while Gartner predicts 40% of projects cancelled by 2027. AWS interviewed 154 leaders over 9 months to find why.

•

1:36 — Efficiency vs Productivity vs Value: Three distinct stages: efficiency (same work faster), productivity (more output per person), value (P&L impact). Most projects stall at stage one. Freed capacity must be explicitly assigned to a named growth initiative within the same quarter or it gets reabsorbed.

•

3:29 — Measurement Traps: Adoption rate ≠ value. One firm had 88% adoption but <1 in 5,000 sessions improved work. Five common metric pitfalls; solution: pair every efficiency metric with a quality counter-metric for 90 days. Retire metrics that no one can recall driving a decision.

•

5:11 — Adoption Maturity Levels: Five levels from access to process redesign. The inflection is level 2 (discrete tasks): below it, 54% find AI useful; above level 3, >85%. Top-down mandates fail; peer teaching, leaders using AI, and progressive difficulty drive real adoption. Middle managers need redefinition from controllers to experiment enablers.

•

7:27 — Skills & Talent Pipeline: Training systems built for execution — now AI's strength. Five durable skills: learning agility, problem definition, judgment, systems thinking, curiosity. Junior judgment pipeline broken: repetition that built judgment is automated. Mechanisms: quarterly 'abandoned capabilities' reviews, cross-domain rotations, promotion criteria for unlearning.

•

9:34 — Governance Mismatch: Six-month governance processes strangle six-day AI experiments, creating shadow AI at 10x shadow IT scale. Principles: agents earn autonomy via probation; guardrails outside the model at system layer; tier controls by consequence severity (decision tree: personal data? real-time? human-in-loop?). Fast lanes for low risk.

•

12:26 — Cost Visibility & Attribution: No historical baseline; costs spike with each capability wave. Transparency enables confidence to spend more, not less. AT&T: 270B tokens/day, every $1 must return >$5 FCF. Costs must sit on the P&L of the team that defines the need and captures value. Platform-enforced rules: budget caps, cheapest viable model default.

•

14:29 — Data Layers & Quality: Two data layers: structured systems and unstructured human context (emails, chats). The second holds undocumented expertise. Data quality isn't 'washed clean' upfront; it's measured by human correction rate in production and improved incidentally while solving real problems. Sequence: start with best data, fix data during projects, then invest in data quality as strategic asset.

Speakers

•

大飞 (Dafei) — Host

⋮