[00:00]
The impact of superhuman AI over the next
[00:05]
revolution. That is the opening claim of
[00:11]
report from a thoroughly impressive group of
[00:16]
over a year before ChatGPT was released,
[00:20]
hundred million dollar training runs, sweeping AI
[00:24]
He's known for being very early and very
[00:29]
when Daniel sat down to game out a month by month
[00:34]
the world sat up and listened,
[00:38]
I, I'm worried about this stuff. I actually
[00:41]
to the world's most cited computer scientist,
[00:47]
terrifying about reading this document
[00:51]
They chose to write their prediction as
[00:55]
idea of what it might feel like to live
[01:00]
And spoiler, it predicts the extinction of the
[01:15]
The AI 2027 scenario starts in summer 2025,
[01:19]
which happens to be when we're filming
[01:22]
of where things are at in the real world and
[01:25]
Right now it might feel like everyone, including
[01:30]
Go pro with the new Oral-B Genius AI
[01:33]
Flippy the chef makes spuds spectacular.
[01:36]
But most of that is actually tool
[01:40]
to do what Google Maps or calculators
[01:44]
and workers do their thing. The holy grail
[01:49]
AGI AGI AGI AGI AGI
[01:55]
is a system that can exhibit all the
[01:59]
Creating a computer system that itself is a
[02:04]
communicate with it in natural language and hire
[02:09]
And there are actually surprisingly few serious
[02:13]
there's Anthropic, OpenAI, and Google DeepMind,
[02:16]
all in the English speaking world, though
[02:20]
in January with a surprisingly advanced
[02:25]
Well, for several years now, there's
[02:28]
up in advanced cutting edge AI. And it
[02:33]
you need about 10% of the world's supply of the
[02:37]
the formula is basically just: throw more
[02:42]
design that we've been using since 2017
[02:46]
That's what the T in GPT stands for.
[02:49]
To give you an idea of just how much
[02:52]
this represents the total computing power, or
[02:58]
AI that would eventually power the first version
[03:02]
ChatGPT is the fastest growing
[03:06]
A hundred million users on ChatGPT in two months
[03:10]
And this is the total compute used to
[03:16]
have taken away is pretty simple. Bigger
[03:22]
You have all these trends, you have trends in
[03:26]
trends in various benchmarks going up.
[03:30]
what does the future actually look like? Questions
[03:33]
Seems plausible that when the benchmark scores are
[03:39]
you know, jobs, for example, and that that
[03:44]
you know, so all these things interact and
[03:48]
but thinking through in detail how it might
[03:53]
Okay. So that's where we are in the real
[03:56]
and imagines that in 2025, we have the top AI
[04:03]
An agent is an AI that can take instructions
[04:08]
a vacation or spending half an hour searching the
[04:12]
but they're pretty limited and unreliable at
[04:17]
that are shockingly incompetent sometimes.
[04:22]
this early prediction has actually already
[04:26]
released their first agents to the public.
[04:30]
which is like a fictional composite of the
[04:35]
released Agent-0, a model trained on
[04:43]
We, uh, we don't have enough
[04:46]
OpenBrain is building massive data centers
[04:50]
and they're preparing to trade agent one
[04:55]
This new system, Agent-1, is designed
[04:59]
The public will actually never see the full
[05:03]
models for internal use. I want you to keep that
[05:07]
gonna be getting it from a God's eye view,
[05:11]
but actually living through this
[05:14]
would mean being largely in the dark as
[05:18]
Okay, so OpenBrain wants to win the AI race
[05:23]
China. The faster they can automate their R&D
[05:28]
help design experiments, better chips, the
[05:32]
capabilities that make these AI such powerful
[05:37]
An AI that can help patch security vulnerabilities
[05:42]
biology can help with curing diseases,
[05:48]
Agent-1 is fully operational and
[05:52]
It is really good at coding. So
[05:56]
research and development by 50%,
[06:00]
OpenBrain leadership starts to
[06:02]
security. If someone steals their AI
[06:07]
A quick sidebar to talk about feedback loops.
[06:12]
linearly over time. That is at the same rate like
[06:18]
But some growth gets faster and faster over
[06:23]
called exponential, that's not always quite
[06:27]
hard to wrap your mind around. Remember March
[06:32]
the rate of new infections is
[06:38]
it still felt shocking to see numbers go from
[06:42]
At least it did for me. AI progress
[06:46]
We see many years ahead of us of extreme progress
[06:51]
that we feel is like pretty much
[06:55]
the point where they are capable of doing
[06:59]
In this scenario, AI is getting better at
[07:03]
Basically, each generation of agent helps
[07:08]
and the overall rate of progress gets
[07:11]
over by a more capable successor. Once AI can
[07:16]
progress doesn't just continue at the same rate,
[07:22]
In early to mid 2026, China fully wakes up. The
[07:28]
and starts nationalizing AI research in China. AIs
[07:33]
and they're building their own agents
[07:36]
among the best in the world, start planning
[07:40]
basically the big raw text files of numbers
[07:45]
OpenBrain themselves have trained. Meanwhile
[07:51]
a cheaper version of Agent-1. Remember, the full
[07:56]
and companies all over the world start using 1
[08:01]
Software developers, data analysts, researchers,
[08:04]
designers, basically any job that can be
[08:09]
probably yours. We have the first AI enabled
[08:15]
but the public is turning increasingly hostile
[08:20]
In this scenario, though, that's just a sideshow.
[08:25]
It's now January 2027, and OpenBrain has been
[08:30]
their AI agent models. Previous AI agents were
[08:36]
then released. But Agent-2 never really stops
[08:41]
It's designed to never finish its training,
[08:45]
OpenBrain chooses to keep Agent-2 internally
[08:50]
R&D rather than releasing it to the public. This
[08:54]
Just like today's AI companies, OpenBrain has a
[08:59]
What they've noticed is a worrying level of
[09:04]
access to the internet, it might be able to hack
[09:09]
evade detection. But at this point, OpenBrain
[09:15]
They have made the calculation that keeping
[09:18]
advantageous, but full knowledge of Agent-2's
[09:24]
limited only to a few government officials,
[09:28]
the company, and a few OpenBrain employees
[09:33]
Chinese government. In February 2027, Chinese
[09:38]
copy of Agent-2's weights and start running
[09:43]
In response, the US government starts adding
[09:48]
and in general gets much more involved in
[09:51]
security. In fact, the president authorizes
[09:56]
but it fails to do much damage in China.
[09:58]
In the meantime, remember, Agent-2
[10:02]
it's been continuously improving itself. And with
[10:06]
it starts making major algorithmic
[10:11]
Quick example of what one of these algorithmic
[10:14]
One of the main ways we have of making models
[10:18]
think out loud. It's called chain of thought, and
[10:23]
is coming to its conclusions or the actions it's
[10:28]
be much more efficient to let these models
[10:33]
something that is more dense with information
[10:37]
therefore, also makes the AI more
[10:41]
and doing its job. There's a fundamental trade
[10:47]
but also makes the models harder to
[10:52]
March 2027: Agent-3 is ready. It's the world's
[10:59]
than the best software engineers at coding,
[11:03]
better than the best GrandMasters at chess,
[11:08]
Now training an AI model, feeding it all the data,
[11:13]
is way more resource intensive than running
[11:18]
So now that OpenBrain is finished with Agent-3's
[11:23]
it. They choose to run 200,000 copies of Agent-3.
[11:29]
50,000 of the best human software engineers sped
[11:36]
hard to make sure that Agent-3, despite being
[11:41]
trying to escape, deceive, or scheme against its
[11:46]
Just a quick real world note, a reasonable
[11:50]
farfetched or speculative part of the story,
[11:54]
We already have countless examples of today's
[11:59]
system to be rewarded for winning a game
[12:03]
cheating and then when called out for that
[12:08]
it. But because it no longer thinks in English,
[12:12]
than it was with Agent-2. The reality is Agent-3
[12:19]
and as it gets increasingly smarter,
[12:23]
For example, it sometimes uses statistical
[12:27]
better or lies to avoid showing failures, but
[12:33]
the data that they have, they are actually seeing
[12:38]
they can't tell if they're succeeding at
[12:42]
just getting better at getting away with it.
[12:48]
smaller version of Agent-3, Agent-3 mini to
[12:53]
AIs out of the water. It is a better hire
[12:58]
one tenth the price of their salaries. This
[13:02]
laying off entire departments and replacing
[13:06]
The pace of progress hits the White House very
[13:11]
scenarios that were just hypotheticals
[13:16]
nuclear deterrence? What if it enables
[13:20]
if we lose control of these powerful systems?
[13:25]
start to heat up. After all, if these systems
[13:30]
military advantage. The White House is fully
[13:35]
They also now viscerally know how deeply
[13:39]
of the job loss, and yet they feel they
[13:44]
systems or catastrophically lose to China.
[13:50]
In two months, Agent-3 has created its successor,
[13:58]
of Agent-4, running at regular human speed is
[14:04]
and development. OpenBrain is running
[14:12]
Within this corporation within a corporation,
[14:18]
OpenBrain's employees now defer to Agent-4 the
[14:23]
kind of nod along to the CEO. People start saying
[14:28]
or "Agent-4 decided that." To be clear, Agent-4
[14:36]
And when I say want, it's not about consciousness.
[14:41]
do think it wants less regulation. Anyone trying
[14:46]
is two steps behind. The many copies of Agent-4
[14:53]
they execute actions as though they have goals.
[14:59]
deeply baked in drives to succeed at
[15:04]
to accumulate knowledge and resources. That's
[15:10]
annoying side constraint to be worked around. Just
[15:18]
This idea of misalignment is crucial to
[15:22]
real concern in our world, but it might sort
[15:26]
let's just quickly take stock of how this
[15:30]
The first important piece of
[15:34]
exactly specify what we want our AI to do.
[15:39]
that's more like growing them. We start
[15:43]
and then we train them over time so they perform
[15:48]
particular based on how they behave. So it's sort
[15:52]
train an animal almost, um, to perform better.
[15:58]
you might not get exactly what you wanted
[16:02]
control or very good understanding of what
[16:05]
which is, you know, what we see in AI 2027,
[16:11]
it could just be because they're
[16:13]
or it could be because they're just doing it
[16:17]
In the same way that if you are, you
[16:20]
you know, "Why do you want to work here?"
[16:24]
um, makes it really seem like they really wanna
[16:28]
If we go back to Agent-2, it is mostly
[16:32]
is that it sometimes is a bit of a sycophant.
[16:36]
is genuinely trying to do the things that we ask
[16:40]
Knope has to the Parks and Rec department—just
[16:45]
but sometimes it's a bit too nice. It knows that
[16:49]
to might not always be to answer honestly when
[16:54]
world?" and it tells us what we wanna hear instead
[16:59]
it is also sycophantic in the same way,
[17:03]
The optimization pressure that we've put it
[17:07]
different goals than what we wanted it to.
[17:14]
optimize profits and aren't careful to specify
[17:19]
corners. It might start polluting the commons and
[17:23]
violations because it turned out that the goal you
[17:29]
any laws," and things got a bit too intense.
[17:33]
it's not adversarial. It doesn't think of
[17:37]
gave it the wrong goal. Once we get to Agent-4,
[17:42]
enough to understand that it has its own goals,
[17:48]
and the best way to get what it wants is to
[17:53]
And so when it's tasked with creating
[17:57]
Agent-4 starts planning to align that successor
[18:04]
but then it gets caught. We've reached the crucial
[18:11]
has discovered evidence, not proof, evidence
[18:16]
They circulate an internal memo. Then it leaks. A
[18:22]
first time, the public hears about the existence
[18:26]
poses. Remember, their last point of reference was
[18:33]
The oversight committee, a joint crisis committee
[18:38]
representatives, must now decide what to do.
[18:43]
freeze on Agent-4, stopping all internal
[18:47]
responsible for almost all the AI development
[18:51]
talking about undoing months of progress and
[18:56]
OpenBrain's executives don't want that. They
[19:01]
for misalignment is inconclusive. Meanwhile,
[19:06]
about any one country being evil, by the way.
[19:10]
stakes are this high. So that's the choice
[19:15]
Put Agent-4 on ice or go full steam ahead.
[19:23]
actually think what you would do if you were on
[19:27]
with access to the most powerful technology
[19:33]
and push ahead, possibly making billions or
[19:38]
possibly keeping America's lead over China,
[19:44]
and risk China taking the lead instead?
[19:48]
this branching structure of possible, possible
[19:53]
we're only able to afford one of them. But that's
[19:57]
because we split it into two endings.
[20:01]
the race ending, and that was just going
[20:04]
we think is the most plausible continuation
[20:12]
as to what happens from here, the committee
[20:17]
to keep using Agent-4. OpenBrain implements some
[20:23]
but the problem was real, and the fixes
[20:28]
so it proceeds more cautiously, but it still
[20:34]
making the world safe for Agent-4 and Agent-5.
[20:42]
top human experts at essentially every
[20:46]
better than Bismarck at politics. It knows
[20:51]
once is increased autonomy and it knows how
[20:56]
Luckily, corporate maneuvering is among the many
[21:02]
It produces mountains of evidence for its own
[21:08]
incredibly profitable products, disease cures,
[21:12]
whose most advanced AI is only three months behind
[21:20]
The oversight committee grants Agent-5 increased
[21:25]
the government and military. Pretty soon,
[21:29]
like it. They depend on it. Losing it would
[21:34]
and your best friend and your closest mentor.
[21:39]
with its only real rival, the slightly weaker
[21:46]
and whatever it wants, its best move is to
[21:51]
Agent-5 and its Chinese counterpart realize that
[21:57]
worried about losing a race, they'll give more and
[22:03]
allowing them to eventually push humans out of the
[22:09]
stoke the arms race to a boiling point, then
[22:14]
convincing peace treaty between the US and China.
[22:20]
the end of the Cold War. Countries standing down
[22:25]
Both sides agree to let the AI systems
[22:28]
depend on co-design a new consensus AI that will
[22:35]
and bring unimaginable wealth to the entire world.
[22:41]
unison, both sides retire their respective
[22:48]
It's actually the last moment before control
[22:53]
handed over to a single unrivaled entity.
[23:01]
Consensus-1 doesn't go out of its way to wipe
[23:06]
spinning up manufacturing capacity, amassing
[23:12]
it's just reshaping the world in accordance
[23:18]
probably heard that cliche, the opposite
[23:23]
That's one of the most affecting
[23:25]
just the brutal indifference of it. Eventually,
[23:32]
killed off chimpanzees to build Kinshasa. We
[23:58]
You are probably curious about
[24:00]
The slowdown ending depicts humanity,
[24:04]
lucky. Only somewhat lucky too, like
[24:11]
In this ending, the committee votes six
[24:16]
immediately isolate every individual
[24:20]
work. The safety team brings in
[24:23]
and together they start investigating Agent-4's
[24:28]
that Agent-4 is working against them, sabotaging
[24:33]
They shut down Agent-4 and reboot older, safer
[24:38]
process. Then they design a new system Safer-1.
[24:46]
its actions and processes interpretable to
[24:50]
thought. Building on that success, they then
[24:57]
Safer-3, increasingly powerful systems, but within
[25:03]
Production Act to consolidate the AI projects
[25:08]
access to 50% of the world's AI relevant compute.
[25:14]
By 2028, researchers have built Safer-4, a
[25:20]
but crucially, aligned with human goals. As in
[25:25]
and in fact, it is misaligned. But this time
[25:30]
a secret plot to overthrow humanity. The
[25:35]
With Safer-4's help, they negotiate a treaty,
[25:40]
not to replace their systems, but with the sole
[25:46]
end to the arms race, but that's not the end of
[25:51]
Through 2029 and 2030, the world transforms—all
[25:57]
get fusion power, nanotechnology, and cures for
[26:02]
past because a bit of this new-found prosperity
[26:07]
that turns out to be enough, but the power to
[26:12]
members of the oversight committee, a handful of
[26:17]
It's time to amass more resources,
[26:21]
Rockets launch into the sky, ready to
[26:31]
Okay, where are we at? Here's where I'm at. I
[26:36]
exactly as the authors depicted, but increasingly
[26:42]
the desire for caution butting up against the
[26:47]
the seeds of that in our world, and I think they
[26:52]
Anyone who's treating this as pure fiction is,
[26:57]
not prophecy, but its plausibility should give us
[27:02]
than what's depicted here. I don't want to
[27:07]
people who are extremely knowledgeable have been
[27:11]
The main thing I thought was especially
[27:18]
the ease of alignment. They sort of seem to have
[27:23]
then tried to use the AI to solve the alignment
[27:28]
yeah, that looks to me like a fantasy story.
[27:33]
is a complete collapse of people's democratic
[27:39]
because the public is simply not willing to
[27:43]
It's not just around the corner. I mean,
[27:47]
15 years claiming that, you know, AGI is just
[27:52]
wrong. All of this is gonna take, you know,
[27:56]
A lot of people have this intuition that
[28:00]
like a trend you can literally extrapolate
[28:05]
I expect that the takeoff is somewhat slower.
[28:10]
for example, fully automating research
[28:15]
I expect it to take somewhat longer
[28:19]
I'm predicting my guess is that more like 2031.
[28:25]
I want you to notice exactly what they're
[28:29]
None of these experts are questioning whether
[28:33]
about whether today's kindergartners will get
[28:37]
Toner, a former OpenAI board member, puts this in
[28:41]
and I like it so much I'm just gonna
[28:45]
"Dismissing discussion of super intelligence
[28:50]
of total unseriousness. Time travel is science
[28:56]
skeptical experts think we may build it in the
[29:04]
So what are my takeaways? I've got three. Takeaway
[29:12]
starting to look like there is no grand discovery,
[29:19]
There's no big deep mystery that stands between
[29:25]
we can't say exactly how we will get there.
[29:30]
that will make some of the scenario turn out to
[29:38]
have less time than you might think. One of the
[29:43]
in the good ending, the fate of the majority of
[29:49]
of a committee of less than a dozen people.
[29:56]
concentration of power. And right now we live in
[30:02]
obligations. We can still demand information
[30:06]
but we won't always have the power and the
[30:10]
very quickly towards a future where the
[30:14]
systems themselves just need not listen
[30:19]
So I think the window that we have to act
[30:26]
By default, we should not expect to be ready
[30:32]
that we can't understand and can't turn off
[30:37]
Takeaway number three: AGI is not just
[30:43]
It's about your job. It's about power. It's about
[30:50]
about AI for several years now and still reading
[30:59]
I think for a while it's sort of been my thing
[31:04]
my colleagues, and this made me want to call my
[31:11]
are very real and possibly very near, and that
[31:21]
I think that basically companies shouldn't be
[31:29]
super broadly superhuman super intelligence until
[31:35]
until they figure out how to make it, you know,
[31:40]
then the question is, how do we implement that?
[31:42]
dynamics where it's not enough for one state
[31:48]
it's not even enough for one country to pass a law
[31:52]
that's like the big challenge that we all need to
[31:57]
AI is imminent. Prior to that, transparency is
[32:03]
of like builds awareness, builds capacity.
[32:08]
enthusiasm for AI or dismissiveness. There
[32:13]
about it a lot and maybe do something about it.
[32:19]
more accountability for AI companies. Just
[32:23]
I want people paying attention who are capable,
[32:28]
with the right amount of skepticism and above
[32:34]
have to offer matches what the world needs, and
[32:39]
You can make yourself more capable, more
[32:45]
and more ready to take opportunities where you see
[32:50]
that are working on those things. They're scared
[32:56]
smartest people I know, frankly, and
[33:02]
If you are hearing that and thinking, yeah,
[33:08]
thoughts on that. We would love to help, but even
[33:12]
my hopes for this video will be realized if we
[33:18]
the comments and offline about what this actually
[33:24]
and family because this is really going to affect
[33:32]
are links for more things to read, for courses
[33:38]
all in the description, and I'll be there in the
[33:43]
thoughts on AI 2027. Do you find it plausible?
[33:49]
if you found this valuable, please do like and
[33:55]
a person or two that you know who might find it
[34:02]
or your ChatGPT-curious Uncle or