AI is about to upgrade itself forever
35sThe idea of AI achieving recursive self-improvement and potentially rendering humanity obsolete is both terrifying and highly shareable.
▶ Play ClipAnthropic, the company behind Claude, filed for a trillion-dollar IPO while simultaneously proposing a global pause on AI development due to fears of recursive self-improvement. The video explores three possible futures: AI takeover, economic collapse from automation, or a disappointing plateau where AI never lives up to the hype. It also discusses historical precedents and recent evidence questioning AI's actual value.
Anthropic filed for a trillion-dollar IPO and proposed a global AI pause due to recursive self-improvement risks.
OpenAI used similar 'too dangerous' rhetoric before releasing GPT-2 in 2019, which turned out fine.
Claude Mythos is better than human researchers 64% of the time, and AI solved an 80-year-old math problem.
Economists predict an 'AI layoff trap' where automation reduces demand, leading to economic collapse.
MIT study: 95% of enterprise AI projects delivered zero measurable ROI despite $30 billion spending.
"The title is accurate—Anthropic is indeed advocating for a pause, but the video also explores other scenarios, making it slightly exaggerated."
Which company filed for a trillion-dollar IPO and proposed a global AI pause?
Anthropic
What is the main danger Anthropic's think tank warns about?
Recursive self-improvement, where AI rewrites its own code without humans.
1:01
What historical precedent does the video cite for AI companies claiming danger before release?
OpenAI did the same before releasing GPT-2 in 2019.
1:40
According to 'Trust Me Bro benchmarks', how often is Claude Mythos better than a human researcher?
64% of the time, Claude Mythos is better than a human researcher.
2:02
What mathematical problem did OpenAI's AI solve that humans couldn't?
OpenAI disproved a central conjecture in discrete geometry that mathematicians failed to solve for 80 years.
2:13
What is the 'AI layoff trap' described by Boston University economists?
The AI layoff trap: firms automate workers, save costs, but laid-off workers stop spending, reducing demand.
2:38
What solution do the economists propose to avoid the AI layoff trap?
Tax automation like pollution to make it cost more to fire people.
3:14
What percentage of AI projects in the MIT study delivered zero ROI?
95% of projects delivered zero measurable revenue impact or ROI.
4:03
What evidence suggests AI apps are not being used despite increased releases?
The number of new app releases nearly doubled, but app reviews and usage declined.
3:41
Recursive Self-Improvement Danger
Highlights the core fear that AI could upgrade itself without human oversight, leading to loss of control.
1:01AI Solves 80-Year-Old Math Problem
Demonstrates AI's capability to solve problems humans couldn't, showing rapid advancement.
2:13AI Layoff Trap Economic Model
Provides a concrete economic mechanism showing how automation could destroy demand.
2:38Tax Automation Like Pollution
Proposes a novel policy solution to align economic incentives with human welfare.
3:1495% of AI Projects Fail ROI
Challenges the hype by showing most enterprise AI investments yield no measurable return.
4:03[00:00] Last week, Anthropic officially became
[00:02] the Apex Alpha company of the artificial
[00:04] intelligence race with a valuation
[00:07] exceeding OpenAI as they filed to go
[00:09] public with their trillion dollar IPO
[00:11] later this year. If you're a software
[00:12] engineer, this comes as no surprise
[00:14] because Claude has been the best AI
[00:16] programmer for years now. But despite
[00:18] the billions of dollars flowing into
[00:19] this company right now, they also just
[00:21] proposed something that sounds insane.
[00:23] Maybe we should wait a second and pause
[00:25] all AI development because AI is getting
[00:27] dangerously close to recursive
[00:29] self-improvement. And when that happens,
[00:31] the last thing humanity ever builds is
[00:33] the thing that realizes it doesn't need
[00:35] humanity. Even if AI is benevolent and
[00:37] doesn't go rogue and kill us all, a new
[00:39] paper just dropped that believes all
[00:41] rational firms will automate each other
[00:43] into a death spiral anyway. It looks
[00:45] like we're screwed no matter what we do.
[00:47] But not everybody out there is a doomer.
[00:49] And I found some compelling evidence
[00:50] that AI might actually kind of suck. It
[00:53] is June 9th, 2026, and you're watching
[00:55] the code report. Anthropic's in-house
[00:57] think tank just dropped a report, and
[00:59] this is the thesis. AI is getting
[01:01] dangerously close to recursive
[01:02] self-improvement. In other words,
[01:04] they're smart enough to rewrite their
[01:06] code and upgrade themselves in a loop
[01:07] with no humans necessary. The entire
[01:09] industry is all gas and no breaks, and
[01:11] they want everybody to come together and
[01:13] hold hands and create a brake pedal. The
[01:15] problem is that Anthropic can't pause
[01:17] alone. While OpenAI, Deep Mind, and XAI
[01:20] keep sprinting. So, unilateral pausing
[01:21] is off the table. It's either everybody
[01:24] or nobody. And that includes China, by
[01:26] the way. And nobody's really worried
[01:27] about the EU. However, a global pause is
[01:29] a very convenient thing for the market
[01:31] leader to advocate for. Because it
[01:33] doesn't erase Anthropic's lead, it
[01:35] freezes it right as they're about to
[01:36] make billions of dollars with an IPO. If
[01:38] all this sounds familiar, it's not
[01:40] because you're crazy. It's because in
[01:42] 2019, OpenAI did the same thing before
[01:44] the release of GPT2. At first, they said
[01:47] it was too dangerous to release it, just
[01:49] like Anthropic is doing right now with
[01:50] Claude Mythos. But then they released
[01:52] GPT2 and it was totally fine. That was 7
[01:55] years ago, and now it looks like ancient
[01:56] technology. Today, if we are to trust
[01:59] these Trust Me Bro benchmarks, the
[02:00] modern Claude models are far better at
[02:02] research than humans. Like 64% of the
[02:05] time, a Claude mythos is better than a
[02:07] human every time. On top of that, AI
[02:09] researchers are now solving problems
[02:11] that humans haven't been able to. Like
[02:13] OpenAI recently disproved a central
[02:15] conjecture and discrete geometry, which
[02:17] mathematicians have failed to do for the
[02:19] last 80 years. The scary thing is that
[02:21] we're already giving AI access to data
[02:23] centers, robots, and weapons to blow
[02:25] people up. And thanks to predictive
[02:27] programming in Hollywood movies, we all
[02:29] know how that story ends. It's either
[02:30] enslavement like the Matrix or
[02:32] extermination like Terminator. I'd
[02:34] prefer the latter, but there's a
[02:36] possibility for an even dumber outcome.
[02:38] as predicted by economists from Boston
[02:40] University in their paper, the AI layoff
[02:42] trap. At this point, there's been tens
[02:44] of thousands of layoffs in tech thanks
[02:46] to AI. But these economists did some
[02:48] math, and it doesn't look good. Because
[02:50] when a firm automates away a worker with
[02:52] AI, it pockets 100% of the savings. But
[02:55] the laid-off worker is also a customer.
[02:57] And their loss spending doesn't just
[02:59] hurt the firm that fired them, it hurts
[03:00] everyone selling anything. Demand goes
[03:02] down. So the endgame is that firms
[03:05] automate their way to infinite
[03:06] productivity and zero demand. They also
[03:08] argue that things like UBI and
[03:10] upskilling aren't going to work and the
[03:12] only solution is to put a tax on
[03:14] automation. It kind of like the same way
[03:15] we tax pollution, making it cost more to
[03:18] fire people. So the math stops rewarding
[03:20] the AI race. But if there's one thing
[03:21] I've learned about economists, it's that
[03:23] they're wrong pretty much every time. A
[03:25] third possibility is that AI just isn't
[03:27] nearly as good as people think and never
[03:29] will be. This is the Wall-E situation
[03:31] where we keep chasing more and more
[03:33] automation and ultimately destroy the
[03:35] planet by building more and more data
[03:36] centers. One piece of evidence that
[03:38] supports that outcome is that over the
[03:40] last couple years with the rise of
[03:41] Agentic AI, the number of new app
[03:43] releases on the iOS app store is nearly
[03:45] doubled. However, it appears nobody's
[03:47] actually using these apps because app
[03:49] reviews and apps with significant usage
[03:51] are declining. In addition, this 2025
[03:54] report from MIT analyzed over 300
[03:57] enterprises implementing AI. And even
[03:59] though they spent over $30 billion
[04:00] collectively, the end result was that
[04:03] 95% of their projects delivered zero
[04:05] measurable revenue impact or return on
[04:07] investment. That doesn't look good. But
[04:09] luckily, there are tools that can help
[04:11] you avoid failure, like Pioneer, the
[04:13] sponsor of today's video. If you're
[04:15] calling a Frontier model for every LLM
[04:17] request in your app, it's probably
[04:19] burning through a bonfire of tokens just
[04:21] to return generic results. But Pioneer
[04:23] solves this by giving you an inference
[04:25] API that you can plug into your existing
[04:27] LLM setup to handle all model routing
[04:30] and optimization for you. And it will
[04:32] cluster your app's traffic by use case
[04:34] to discover where your current model is
[04:36] being too slow, too expensive, or too
[04:38] stupid. Then it trains a fleet of
[04:40] smaller open- source models in the
[04:42] background and alerts you when it finds
[04:43] one that's cheaper and better so you can
[04:46] easily swap it under the hood. But
[04:47] Pioneer works great with Claude Code,
[04:50] Codeex, Cursor, Hermes, or anything else
[04:52] hitting an LLM endpoint. And you can get
[04:55] $30 of inference for just $5 today. But
[04:58] this has been the code report. Thanks
[04:59] for watching and I will see you in the
[05:01] next one.
⚡ Saved you 0h 05m reading this? Transcribe any YouTube video for free — no signup needed.