Legal System Is Drowning in A.I. Slop

Original Media · 27m · English · YouTube · LegalEagle

TLDR

AI-generated 'slop' has metastasized from internet junk into actual courtrooms, legislation, and judges' chambers. Viral deepfake courtroom clips, AI-hallucinated citations in federal orders, a deceased victim's AI impact statement, and fabricated evidence jailing innocent people illustrate a crisis where legal professionals and courts lack the training and tools to detect synthetic content, risking corrupted precedent and wrongful convictions. Meanwhile, grifters use AI to sell illegal schemes as 'loopholes,' and unrepresented litigants gain false confidence from models that never say 'your claim has no merit.' Courts are beginning to push back, but most lack the expertise to catch sophisticated forgeries.

For you

PRO members can sign in and request more information — or their own version of this digest, tuned to what they care about.

Sign in to unlock

Surprising Insights

•

Most federal judges now use AI tools in chambers, but nearly half received no formal training on them.

•

A federal judge's law clerk used Perplexity AI to draft a restraining order that cited a case that doesn't exist (Cousins v. School Board of Norfolk, 1974).

•

An AI-generated avatar of a deceased crime victim was allowed to deliver a victim impact statement in a real US sentencing hearing, and the judge thanked the AI.

•

AI models never tell unrepresented litigants their claims lack merit — they only validate and elaborate them, flooding courts with confident but meritless filings.

•

A woman was jailed for two days and fought charges for eight months because her boyfriend submitted a single AI-faked text message screenshot.

•

Courts admit they lack the time, funding, and technical expertise to detect less obvious deepfakes, implying many fabricated exhibits go undetected.

Key takeaways

•

Social media is flooded with AI-generated fake courtroom videos — like a tattooed bodybuilder in a Hannibal Lecter mask receiving a 900-year sentence — that exploit engagement algorithms; telltale signs include impossible restraints, dog-style muzzles that don't exist in US courts, and visual artifacts like blurry margins and frame glitching.

•

'Unethical but legal' money-making videos coach viewers to commit crimes like grand theft auto by misrepresenting abandoned-vehicle lien laws; the host debunks a viral claim that swapping a car's battery creates a mechanic's lien, citing a 2000 NYC case requiring owner authorization.

•

A 2026 survey of 112 federal judges found 60% use AI tools in chambers, 22% daily or weekly, yet over 45% received no AI training from court administration; two judges issued error-ridden orders in summer 2025, one citing a non-existent 1974 case (Cousins v. School Board of Norfolk) and another misquoting statutes and inventing declarations.

•

In Jackson Federation of Teachers v. Fitch, Judge Henry Wingate's law clerk used Perplexity as a 'foundational drafting assistant' for a restraining order blocking Mississippi's DEI ban; the order named wrong plaintiffs, cited phantom declarations, and the judge refused to explain, calling errors 'clerical' — now the Fifth Circuit may overturn the ruling and remove him.

•

A pro se litigant, Jerome Dewald, submitted an AI-generated avatar to argue his appeal in New York; the court caught it within seconds when a justice asked, 'Is that counsel for the case?' Dewald admitted he used the avatar to avoid his own 'mumbling, stumbling, and tripping over words.'

•

In Arizona, Chris Pelkey's sister used AI to recreate his voice and image for a victim impact statement at his killer's sentencing; Judge Todd Lang allowed it, saying 'I love that AI,' marking the first known US court use of a deceased person's AI testimony — raising concerns about future misuse by less scrupulous parties.

•

Deepfake evidence has already jailed innocent people: Melissa Simms spent two days in a Florida jail and eight months fighting charges after her boyfriend submitted an AI-faked text screenshot; in Mendones v. Cushman & Wakefield, plaintiffs submitted multiple deepfake videos, doctored Ring photos, and altered chat screenshots with stuffed metadata, prompting dismissal with prejudice.

•

Courts are starting to reject AI-enhanced evidence: in State v. Peluca, a Washington court excluded Topaz Labs AI-enhanced video under the Frye test because forensic video analysts — not video producers — are the relevant scientific community, and the software's opaque algorithm had never been peer-reviewed.

Notable quotes

“A badly governed public large language model never says to an unrepresented party, this claim has no merit. It never tells them they are wrong. It simply tells them they are right, formulates their position elegantly, and drops them into a gridlocked court queue.”

▸ Notable quotes · 17:04

“I love that AI. Thank you for that.”

▸ Notable quotes · 15:40

“Okay, hold on. Is that counsel for the case?”

▸ Notable quotes · 14:20

“I generated that. That's not a real person.”

▸ Notable quotes · 14:22

“The court remains suspicious of other evidentiary submissions, but it does not have the time, funding, or technical expertise to determine the authenticity of plaintiff statements or conduct a forensic analysis of the suspect evidentiary submissions.”

▸ Notable quotes · 23:19

“You know, I think AI will probably, like most likely, sort of lead to the end of the world, but”

▸ Notable quotes · 24:34

Chapters

•

0:00 — AI Courtroom Slop on Social Media: The video opens with examples of viral AI-generated courtroom fakes — bodybuilders in Hannibal Lecter masks, fake 'infinite money glitch' legal advice — and teaches viewers to spot watermarks, frame glitching, legally impossible premises, and visual artifacts like blurry margins.

•

2:12 — Anatomy of a Fake Sentencing Video: Frame-by-frame analysis of a viral 900-year sentencing clip reveals impossible restraints (dog-style muzzle, unknown shackles), non-reactive attorney, proportion errors, and telltale AI generation signs like detailed foreground against grainy background; the clip spawned an army of AI remixes chasing creator-fund payouts.

•

7:25 — 'Unethical But Legal' Money-Making Schemes: Debunks viral videos teaching viewers to steal abandoned luxury cars via fake lien sales or manufacture mechanic's liens by swapping batteries; explains actual Port Authority procedures and a 2000 NYC case requiring owner authorization for any lien.

•

9:15 — Lawyers and Firms Embrace AI Marketing: 63% of lawyers surveyed by Thomson Reuters in 2024 used AI; firms release uncanny-valley ads (shirtless grandpas, custard-eating contests) while the efficiency pitch ignores how AI-generated paperwork floods already-backlogged courts.

•

10:15 — Judges Using AI — And Hallucinating Citations: 2026 survey: 60% of 112 federal judges use AI tools, 45% without training. Judge Neals withdrew an order inventing quotes and misstating case outcomes; Judge Wingate's clerk used Perplexity to draft a DEI-ban restraining order citing a non-existent 1974 case, prompting Fifth Circuit review.

•

13:10 — AI Avatars for Pro Se Litigants and Dead Victims: Jerome Dewald's AI avatar argued his NY appeal until a justice asked 'Is that counsel?' In Arizona, Chris Pelkey's sister generated his AI impact statement; Judge Lang allowed it, saying 'I love that AI' — the first known US court use of a deceased person's AI testimony.

•

20:22 — Deepfake Evidence Jailing Innocent People: Melissa Simms jailed over AI-faked text screenshot; Mendones plaintiffs submitted deepfake videos, doctored Ring photos, altered chat screenshots with stuffed metadata — case dismissed with prejudice as 'egregious breach of trust.' Court admitted it lacks resources to catch less obvious forgeries.

•

23:35 — Courts Push Back on AI-Enhanced Evidence: In State v. Peluca, Washington court excluded Topaz Labs AI-enhanced video under Frye test: forensic video analysts (not video producers) are the relevant community, and the opaque algorithm lacked peer review. Courts are drawing lines but most lack forensic capacity.

Speakers

•

Devin Stone — Host

⋮