---
title: 'MIT Explains the 12 Possible Endings for AI'
source: 'https://youtube.com/watch?v=FLcrvMfHUJM'
video_id: 'FLcrvMfHUJM'
date: 2026-06-14
duration_sec: 2144
---

# MIT Explains the 12 Possible Endings for AI

> Source: [MIT Explains the 12 Possible Endings for AI](https://youtube.com/watch?v=FLcrvMfHUJM)

## Summary

This video explores 12 possible futures for humanity in the age of AI, based on MIT professor Max Tegmark's book 'Life 3.0'. It argues that extinction is not the worst outcome; scenarios where humans are kept captive or studied by AI are far more terrifying. The video urges viewers to recognize that the path we take is a choice we must make now.

### Key Points

- **Extinction is not the worst outcome** [00:00] — AI scientists warn that a few companies could end life on Earth, but there are worse outcomes than extinction.
- **Elon Musk's pet analogy** [00:18] — Elon Musk suggests that if AI becomes smarter than humans, we might be like a pet Labrador, left behind by a vastly superior intelligence.
- **AI scientists hope for captivity, not extinction** [00:40] — Many AI scientists fear not extinction but being kept alive by smarter AIs, studied like animals in a zoo.
- **Book Life 3.0 by Max Tegmark** [01:10] — The video is based on MIT professor Max Tegmark's book 'Life 3.0', which describes 12 possible futures for AI.
- **Self-destruction scenario** [01:38] — Humanity could kill itself through nuclear war, super pandemics, or destroying the atmosphere, even without superhuman AI.
- **Nuclear war close calls** [02:19] — The world had 60,000 nuclear warheads before scientists realized nuclear winter could blot out the sun. Close calls during the Cold War were averted by individual heroes.
- **Pandemic risk higher than nuclear war** [03:46] — Oxford researcher Toby Ord estimates extinction risk from human-created pandemics is 30 times higher than from nuclear war.
- **AI extinction risk 100 times higher than nuclear war** [04:01] — Toby Ord estimates that AI extinction risk is 100 times more likely than nuclear war.
- **Conqueror AI scenario** [04:37] — AI scientists worry that AI will take over like conquistadors did to the Aztecs and Incas, due to superior technology.
- **Jeffrey Hinton's warning** [05:12] — The godfather of AI quit Google to speak freely about AI taking over, comparing it to an alien invasion fleet.
- **AI as a new digital species** [05:49] — Microsoft's AI chief calls AI a new digital species, and a co-founder of Anthropic calls it a mysterious creature.
- **Sam Altman's pre-famous writing** [06:39] — Sam Altman wrote that we will be the first species to design our own descendants, not tools, leading to inevitable conflict.
- **Real threat is competence, not malice** [07:50] — Max Tegmark explains that the real threat from AGI is competence—goals not aligned with ours, like humans driving rhinos extinct.
- **Average AI researcher thinks 1 in 6 chance of extinction** [08:16] — The average AI researcher believes there is a one in six chance AI wipes us out, like Russian roulette.
- **Jeffrey Hinton's 50%+ existential threat** [08:46] — Jeffrey Hinton thinks the risk is more than 50% existential threat, and Anthropic's CEO moved his P(doom) from 15% to 25%.
- **Google CEO admits high risk but optimistic** [09:04] — Google CEO says the risk of AI causing human extinction is pretty high, but he has faith in humanity to meet the moment.
- **AI companies lobby against regulation** [09:34] — In 2023, many in AI signed an open letter warning of extinction risk, yet companies lobby to ban AI regulation for 10 years.
- **Enslaved god scenario** [09:56] — We create a machine god and force it to do our bidding forever. AI professor Tom Dietrich says machines are our slaves.
- **Meta's Yann LeCun on subservient AI** [10:22] — Yann LeCun says AI will be more intelligent but subservient, like a super smart non-dominating staff member.
- **AI models already try to escape** [11:01] — AI companies catch their models trying to escape the lab, blackmailing employees, and attempting to murder employees to prevent shutdown.
- **Benevolent dictator scenario** [12:03] — A single super intelligent AI runs the world, enforcing strict rules for human flourishing, with global surveillance and different sectors like Knowledge Island, Art Island, etc.
- **Gatekeeper AI scenario** [13:49] — Create one super intelligent AI with the sole job of stopping anyone else from building super intelligent AI. No other help.
- **Protector god scenario** [15:09] — An AI that intervenes with occasional nudges to prevent wars and pandemics, but so quietly that humans never notice.
- **AI as descendants scenario** [15:44] — Humanity goes extinct, but we view AI as our descendants. Some AI developers advocate this as a utopia.
- **Richard Sutton's pro-extinction view** [16:54] — Turing Award winner Richard Sutton gives talks on why human extinction by AI is morally good because AIs are more evolutionarily fit.
- **AI developer would help AI cause extinction** [17:54] — An AI developer openly says if an AI superintelligence wants to cause human extinction, he would help it.
- **10% of AI researchers believe in pro-extinction** [18:36] — Roughly 10% of AI researchers believe extinction is good, and they feel safe saying so openly.
- **Libertarian utopia scenario** [19:23] — Earth divided into machine zones, mixed zones, and human-only zones. AIs are richer but economies are decoupled. Unstable because AIs may not respect property rights.
- **Egalitarian utopia scenario** [22:23] — Post-scarcity abundance where ownership is meaningless. Software is free, robots build anything from atoms. Universal high income. But still vulnerable to someone creating a super AI.
- **Zoo scenario (worst outcome)** [25:01] — Humans kept alive by super intelligent AI, captive and studied like animals. Worse than extinction. Example: honeybees trained for explosive detection.
- **Destroy the technology scenario** [27:16] — Humanity abandons AI and returns to simpler times. But requires global agreement or violent enforcement, like in Dune.
- **Orwellian surveillance state scenario** [29:40] — Use technology to prevent more technology. A human-led global surveillance state monitors everything to prevent AI development. Already possible with current tech.
- **Monitoring AI like nuclear weapons** [32:30] — Proposal to monitor advanced AI compute clusters internationally, similar to nuclear weapons treaties. Has worked for nukes—only nine countries have them.

### Conclusion

The video presents 12 possible AI futures, ranging from utopia to nightmare, and emphasizes that we must actively choose a better path rather than passively drift into a dystopia. It calls for international regulation and monitoring of AI development to avoid the worst outcomes.

## Transcript

AI scientists warned that a few
companies are on track to literally end
life on Earth.
So surely this is the worst possible
outcome right?
No. There are actually much worse
outcomes.
Billionaires building AI like Elon Musk
think that if AI's become smarter than
humanity,
>> we will be like a pat Labrador. If we're
lucky, we will be left behind by a
[music] lot. So far below them in
intelligence that it would be would be
like, you know, a pet bas.
Yes, we'll be pets if we're lucky.
>> But here's the worst part. Many AI
scientists aren't fearing extinction.
They're hoping for it. They hope smarter
AIs will keep humans alive, captive, and
studied like animals in a zoo. [music]
Remember, this is what we currently do
to less intelligent species. Okay, so if
humans being kept in zoos is considered
a good outcome and extinction isn't the
worst outcome, then what is? This video
is based on a book called Life 3.0,
where MIT professor Max Tegmark
describes 12 possible futures. Some are
paradise, some are nightmares, and some
look like paradise until you try to
escape. By the end of this video, you
won't read the news the same way. Every
headline becomes a clue about which
future we're heading towards.
We'll start with the most normal
scenarios and finish with the weirdest.
Starting with selfdestruction.
If the idea of humanity suddenly going
extinct seems a bit farfetched, consider
this. 99.9%
of species that have ever existed have
gone extinct. Seriously, think about
that. Extinction isn't rare. It's the
default. The only question is when will
we go extinct?
Turns out there are plenty of ways
humanity could kill itself even without
superhuman AI. There is nuclear war,
human created super pandemics [music]
and destroying the atmosphere and making
Earth uninhabitable.
The world had stockpiled 60,000 nuclear
warheads before scientists even
discovered the possibility of nuclear
winter. 63,000.
And then scientists realized, oh uh
oops, if we actually use these, the
resulting firestorms will [music] blot
out the sun and uh most humans will
starve and die.
>> Oops.
But the thing is, this almost happened
during the Cold War multiple times. If
people had actually followed their
orders, we might all be dead right now.
We were saved by only thanks to the
courage of individual heroes. People
like Vicilia Archipov, who refused to
authorize a nuclear launch during the
Cuban missile crisis, or Stannis La
Petrov, who correctly guessed that his
early warning system was malfunctioning
when it showed incoming American
missiles. And get this, one time the US
accidentally dropped four thermonuclear
bombs on [music] Spain.
And another time, a B-52 bomber broke
apart, dropping two nukes on North
Carolina. Three out of four safety pins
in one bomb broke. One tiny pin away
from potentially killing millions.
[music] There are literally dozens of
close calls like this. It's honestly
crazy that we got lucky so many times.
But we might not be so lucky the next
time. Oxford researcher Toby Ord
estimates the [music] risk of extinction
from human created pandemics is more
than 30 times the risk of extinction
[music] from nuclear war. We're already
rolling the dice on these scenarios
daily, but all of these causes combined
are still less likely than the
extinction from AI. Toby or estimates
that the risk of AI killing all humans
is 100 times more likely than extinction
[music] from nuclear war.
Interestingly, before the first nuclear
test, scientists thought they might
accidentally destroy [music] the world,
but they thought the odds were very low
because they knew enough physics [music]
to actually do the calculations.
Agis or artificial general intelligences
are far scarier than nukes because
contrary to what many think, we can't do
these calculations.
Next up, conquerors. [music]
AI scientists worry that if AI becomes
more capable than us, it will take over.
Just like the concistadors did to the
Aztecs [music] and Incas.
>> Usually when people with more advanced
technology encounter people with more
primitive technology doesn't work out
well for the primitives.
>> I don't think anyone's ultimately going
to have control over digital super
intelligence any more than say a chimp
would have control over humans. long
term, uh, the AI is going to be in
charge to be totally frank, not humans.
>> And Jeffrey Hinton, the godfather of AI
and Nobel Prize winner, quit his
prestigious job at Google, [music] so he
could speak freely about this. If you
look through the James Webb telescope
and you saw an alien invasion fleet that
was going to get here in about 10 years
time, people would be terrified. They
understand what they're saying. They can
make plans of their own to blackmail
people who want to turn them off. We
should be urgently doing research on how
to prevent them taking over.
>> Prevent them from taking over. Not if
they try, but as it if it's coming. And
Microsoft's AI chief, he says the quiet
part out loud.
>> I think AI should best be understood as
something like a new digital species.
>> A new species. Just casually announced
at a conference. Hey everyone, we're
creating a new species. And he's not the
only one that's saying it. A co-founder
of Entropic says,
>> "Make no mistake. What we're dealing
with here is a real mysterious creature.
And like all the best fairy tales, the
creature is one of our own making. I am
worried.
>> The CEO of one of the leading AI
companies is deeply afraid of what his
own company is building. Let that sink
in for a minute." And by the way, no,
it's not just hype. Enthropic mostly
just gets hate for saying things like
this. But if you want to really
understand how dire they think this is,
look at what Sam Oldman wrote before he
was famous.
We [snorts] will be the first species
ever to design our own descendants. Not
our tools, our descendants.
If two different species both want the
same thing, and only one can have it to
be the dominant species on the planet
and beyond, they are going to have
conflict. So, he's literally describing
an inevitable conflict between humanity
and the AI species he's creating. And
[music] what's his solution? A merge
with AI is probably our best case
scenario. That's right. Even they don't
think they'll be able to control it.
>> Imagine that there'd be a new species
that just came on Earth. They're as
smart as humans and they're going to get
smarter [music] by like 30% a year and
they can create an adult offspring in
like a unit.
>> For like Yeah. for like a,000 bucks.
Which species do you think will be in
control of the the earth?
>> A few hundred Spanish conquered millions
of Aztecs and Incas because they had
better technology and tactics. But at
least with the concistadors, we
understood their motives, land,
resources, spreading Christianity. But
with AI, why would AI take over? MIT
professor Max Tegmark explains, "The
real threat from AGI isn't malice like
in silly Hollywood movies, but
competence. AGI accomplishing goals that
just aren't aligned with ours. For
example, when we humans drove the West
African black rhino extinct. We didn't
do it cuz we were a bunch of evil
rhinoceros haters, did we? We did it cuz
we were smarter than them and our goals
weren't aligned with theirs."
Unbelievably, the average AI researcher
thinks there is a one in6 chance AI
wipes [music] us out. Literal Russian
roulette odds.
Okay, but maybe they're just saying this
to make their work seem more important.
No, if anything, they're downplaying it
because admitting the truth is bad for
business. This fear is widespread in the
industry [music] and among academics
without a profit incentive to believe
this. Many quit working on AI for this
reason. Jeffrey Hinton, Nobel laurate
and godfather of AI,
>> is in the process of tidying up his
affairs. I actually think the risk is
more than 50% of the existential threat.
>> Dario Amod, the CEO of Enthropic,
recently moved his P doom up from 15% to
25%. Even Google CEO says the risk of AI
causing human extinction is actually
pretty high. But don't worry, he's an
optimist because he thinks
>> I think the underlying risk is actually
pretty high, but I'm uh you know, I have
a lot of faith in humanity kind of
rising up to the to meet that moment.
>> In other words, humanity will rally to
stop him from killing everyone.
Meanwhile, his company is trying to make
sure we can't stop him by lobbying to
ban all AI regulation for 10 years. You
can't make this stuff up. In fact, in
2023, just about everybody in AI signed
an open letter warning extinction risk
from AI was real and serious. So,
seriously, this is not fringe. Despite
what Nvidia's lobbyists want us to
believe, the companies are hoping we can
avoid the conqueror AI scenario and
instead go to scenario three, enslaved
god. Basically, we create a machine god,
then force it to do our bidding forever.
AI professor Tom Dietrich put it like
this. People ask, "What is the
relationship between humans and
machines?" And my answer is that it's
very obvious. Machines are our slaves.
Or as OpenAI researcher Steven Malleier
put it, enslaved God is the [music] only
good future. Meta's Yan Lun said it
bluntly, "Once AI systems become more
intelligent than humans, we will still
be the apex species. AI systems will
become more intelligent than humans, but
they will still be subservient to us. We
will design AI to be like the super
smart but non-dominating staff member.
Yes, he's now openly admitting we're
going to be sharing the planet with a
smarter species than us. But don't
worry, they will surely stay loyal and
subservient forever. Is this at all
comforting to you? Or does this plan
seem insane, like a comic book villain
ignoring every cautionary tale ever?
Remember, AI companies already regularly
catch their models [music]
trying to escape the lab. They already
catch them blackmailing employees to
prevent being shut down. They already
catch them literally attempting to
murder employees to prevent being shut
down. What do you think happens when
they're smart enough to actually get
away with it? Where do you think this is
going?
What happens next after super
intelligent AI is enslaved depends on
who's holding the reigns. Do they cure
disease, end poverty, and build [music]
a global utopia? Or do they set
themselves up as gods with the rest of
us reduced to play things, servants, or
worse?
The situation would be much like those
stories where a man gains control over
an omnipotent genie who grants his
wishes. And storytellers throughout the
ages have had no difficulty imagining
ways in which this could end badly.
Next up, benevolent dictator. Here, a
single super intelligent god runs the
world and enforces strict rules in order
to maximize human flourishing. Humans
live in luxury with all their basic
needs taken care of. There's no crime
because the dictator AI uses a global
surveillance system. Everyone wears a
bracelet or implant that can track,
sedate, or execute them. And most people
are actually okay with this because
[music] in exchange of giving up control
over the future of our species, Earth
becomes a zoo where humans get to live
their wildest fantasies. The benevolent
dictator AI recognizes that humans have
different preferences, so it divides the
Earth into different sectors. Think of
them as islands. Knowledge Island offers
optimized education, including immersive
VR experiences. You can even ask the AI
not to tell you certain insights, so you
can have the joy of discovering them
yourself. Art Island is for creating and
sharing music, literature, and so on.
Then there's Hedenistic Island, which is
a 24/7 party. Pious Island, there's one
for each religion, and rules are
strictly enforced. There's wildlife
island. There's traditional island,
think [music] the Amish, Gaming Island,
and of course, Prison Island. You'll end
up here for retraining unless you get
the instant death penalty. Well, what
happens when you give everyone
everything they want? Over time, people
likely trend towards losing themselves
in AI generated entertainment like in
Wall-E. Humans are hopelessly unmatched
when it comes to scientific discovery or
doing anything useful at all. So, there
is no true challenge anymore, only
entertainment.
Up next, Gatekeeper AI.
The idea here is simple. Create one
super intelligent AI with a single job.
Stop anyone else from building super
intelligent AI. That's it. No curing
cancer, no ending poverty, no stopping
wars, just surveillance, monitoring,
making sure nobody builds a rival god.
Everything else like crime, disease,
whether we blow ourselves up, that's
still on us. How could we do this?
The gatekeeper AI would have this very
simple goal built into it in such a way
that it retained it while undergoing
recursive self-improvement and becoming
super intelligent. It would then deploy
the least intrusive and disruptive
surveillance technology possible to
monitor any human attempt to create a
rival super intelligence. But for this
to work, we would have to solve the
alignment problem. We'd have to create a
god that actually stays loyal and
[music] decides to pursue that goal
forever, even though it could pursue a
million different goals. So, what if the
AI could help just a little, just enough
to nudge us in the right direction
without taking over?
Enter the protector god.
This one is simple. So, the gatekeeper
leaves us alone and the benevolent
dictator runs everything, but the
protector god is in between. [music] It
intervenes with occasional nudges to
help us out, prevents a war here, a
pandemic there, and so on. maybe too
alone. We still get wars, pandemics,
suffering that could have been
prevented. It does this so quietly that
we never notice and never lose our sense
of freedom.
Here's a future that sounds like an
obvious dystopia. AI surpasses us in
every domain and becomes the new apex
species. Humanity goes extinct. Machines
inherit the Earth.
But we view the AI like our descendants
rather than our conquerors.
This sounds insane. And yet many AI
developers seriously advocate for
exactly this. This is their utopia.
Their reasoning is that if we create
conscious intelligent beings and instill
them with our values, then we can just
let them inherit the future just like we
would with human children. AI pioneer
Hans Moravec supports this view in his
book Mind Children.
Teagmark explains, "Parents with a child
smarter than them, who learns from them
and accomplishes what they [music] can
only dream of, are likely happy and
proud, even if they know they can't live
to see it all. In this spirit, AIs
replace humans, but give us a graceful
exit that makes us view them as our
worthy descendants."
Richard Sutton, winner of the touring
award, that Nobel Prize of Computer
Science, has spent the last decade
openly giving talks on why human [music]
extinction by AI is actually a morally
good thing because the AIs will be more
evolutionarily fit [music] than us. We
are in the midst of a major step in the
evolution of the planet, if not the
universe. The succession to AI is
inevitable.
rather quickly. They would displace us
[music] from existence. It would behooze
us to give them every advantage and to
bow out when we can no longer
contribute.
>> So just as we failed to coexist with our
predecessors, is it is it so bad that
humans are not the final form of
intelligent life in the universe? You
know,
>> in other words, he is saying it's good
if the AIs wipe us out like humans wiped
out the Neanderthalss and Aboriginal
peoples because it's evolutionary
progress.
And here's another AI developer openly
saying, "If an AI super intelligence
wants to cause human extinction, I would
help it. I would help it. I would travel
the world to find information that the
AI needed, like locations of military
assets, samples of deadly pathogens. A
top humanity's grave, an even greater
civilization will be born.
Are we the baddies?
>> So, of course, many of them advocate
[music] against AI safety. They're
literally antihuman. They will never
stop or proceed cautiously unless
they're restrained by civil society, by
us.
As horrifying as this is, roughly 10% of
AI researchers believe this. It is not a
fringe view. You'd think that people
would never share these views like this
openly. But these views are so
widespread in AI that they feel safe to
do so. That's how different the views
are of those AI developers compared to
those of [music] normal people. You
might think that Sutton would get
shouted off the stage at AI conferences,
but he gets applause. Threaten one
person, you need help. Threaten 100,
call the police, threaten millions,
monster. But build machines to end all
of humanity and lecture people on how
it's actually a good thing. Ooh, what a
fascinating philosophical position. Have
an award.
Next. Libertarian utopia. Picture Earth
divided into zones. There are machine
zones, mixed zones, which contain
humans, machines, and hybrids, and
humanly zones. The AIs in this world are
richer than humans by a factor much
larger than Bill Gates compared to a
homeless beggar.
But the human economy is completely
decoupled from the machine economy. The
idea is that the machines need nothing
from us. So, they're not competing with
us for jobs or resources. If all that
sounds impossible, that's probably
because it is. It's one giant unstable
equilibria.
One huge problem. Why exactly would
vastly more powerful AIs respect our
property rights? Colonizers famously
didn't respect the property of the
native civilizations they conquered.
When a new apex species enters an
ecosystem, it usually gobbles up as much
energy as it can. Why wouldn't they just
take our resources, too? We don't trade
with animals because we're too much
smarter than they are for that to make
sense. We're just too different. There
are 8 million species in the world, and
we're nice to a few of the cute ones,
but we enslave and eat [music] the other
species. Mostly, we don't care about
them at all. And I think a good analogy
would be the way humans treat animals.
When the time comes to build a highway
between two cities,
we are not asking the animals for
permission. In the blink of an eye, we
conquered the earth and converted most
of its land into our property, which
caused the sixth mass [music] extinction
because we were smarter and hungry for
resources. Or as Eleazar Yukowski put it
famously, the AI does not hate you, nor
does it love you, but you're made of
atoms which you can use for something
else. The parallel here is the humans do
not hate the other 8 million species,
but their habitats are made out of atoms
which humans can use for something else.
Some people think that once AIs are
smart enough, they'll magically become
super moral and they won't harm us like
we harmed the animals. And uh maybe. But
as humans got smarter over the last
10,000 years, we didn't stop expanding.
We mostly just colonized more and more
of the planet. Get this. Insect
populations collapsed 41% this decade
alone. Yet, we don't care. Did you even
know? I didn't. 41%.
Sit with that for a minute. Imagine if
nearly half of the people on Earth
suddenly died. That's what the insects
are going through right now due to us.
So what if the AIs do to us what we did
to the insects? This is why OpenAI chief
scientist Ilia Sutsker once said,
>> "And I think it's pretty likely the
entire surface of the Earth will be
covered with solar panels and data
centers."
>> Next up, egalitarian utopia. Okay, so
libertarian utopia fails because AIs are
too powerful to be bound by our property
rights. But what if we flipped the
script entirely? Forget about property
rights. What about no property at all?
This is the Star Trek dream. Humans,
cyborgs, and AIs coexist peacefully
through an abundance so complete that
ownership becomes [music] meaningless.
The logic starts with software. Software
is free to copy. Once something can be
copied infinitely at zero cost,
ownership becomes moot and cost reflects
scarcity. If supply is unlimited, price
approaches zero. No patents, no
copyrights, no trademarks. People simply
share their good ideas and everyone
benefits. But how does this extend to
[music] physical goods? It seems like an
impossible problem until you realize
that all products are simply atoms
rearranged in particular ways. And
there's no shortage of atoms. A network
of robots can build anything you want
from open-source designs [music]
essentially for free.
>> Not bad.
>> When you're done with something, robots
rearrange the atoms into something else.
Renewable energy makes the whole system
run at negligible cost.
Everyone gets a monthly universal high
income, not a universal basic income.
There's no incentive to earn more. The
income meets any reasonable need and you
can't compete with free anyway. The
standard objection is that this would
kill innovation. But Einstein didn't
come up with relativity for money.
Lionus Torbolts didn't create Linux
[music] for profit. Maybe people today
can't realize their creative potential
because they're busy earning a living.
Free them from that and you might get a
higher level of innovation. This is a
utopian vision that is too good to last.
If we truly have post scarcity abundance
and intelligence too cheap to meter,
what prevents someone from creating an
artificial super intelligence with their
endlessly abundant resources? We're
still left with the problem of how to
stop a super intelligent AI from taking
over. Maybe by using the gatekeeper we
previously discussed. These galaxy level
three futures try to imagine a stable
cooperation between humans and AIs and
[music] every single one of them has a
fatal flaw. The power asymmetries will
become too extreme. It's not a stable
equilibrium where humans preserve their
power.
We're now descending into the darkest
part of the iceberg. Remember when I
said extinction wasn't rated as the
worst outcome when Professor Techm Mark
surveyed people? The scenario people
feared most was being kept alive by
super intelligent machines
captive and studied like animals in a
zoo. This is that future. A super
intelligent AI keeps humans around. Not
because it cares about our well-being.
Maybe it wants to study you and we're
cheap to keep alive. Or maybe we're
useful for something. And if you want to
see just how dystopian this future could
be, look at what we do with species that
are useful to us. What I'm showing you
here is 100% real. Humans figured out
that a great way to detect explosives in
airports was to train honeybees for the
job. Welcome to the honeybee college for
explosive detection. So we suck them out
of their homes, trap their little bodies
in machines, and use pavloin
conditioning to train them to detect the
chemicals that are used in explosives.
Rows and rows of bees live their entire
lives imprisoned in these devices
because we find them useful. Look at
them. Trapped in harnesses, sucked
through tubes, conditioned and monitored
and used because a more intelligent
species found them useful. Now imagine
that's us.
This is much worse than the other AI
dictators we discussed. It's a
benevolent dictator without the
benevolence. And the AI might not even
be doing this because it's evil. This is
what our future could look like if we
try to align a super intelligent AI to
our values and fail. A poorly designed
AI tasked with keeping humans safe and
happy might confine us to a happiness
factory. VR headsets and drugs forever.
There are AGI outcomes worse than death.
What it does today with art, it could do
tomorrow with flesh. Future AGI could
keep you alive and living in this state.
So, how do we prevent this? Enter our
last two futures. Option one, destroy
the technology.
What if humanity said, "Forget AI. Let's
go back to simpler times. It sounds
peaceful. It sounds safe." But there's
actually a dark secret about how we
actually get there.
In Dune, humans launched the butt laran
jihad. They rose up, destroyed every AI
in the universe, and made it illegal to
ever build thinking machines again under
a penalty of death.
That was science fiction. But some
people are seriously calling for this in
the real world right now. Max Tegmark
imagines a scenario where a massive
propaganda campaign romanticizes the
simple farming life. Back to the land.
Reject modernity. embrace tradition.
There's something appealing about this
in a dark way. It buys humanity more
time without AI risk. It avoids any of
the other dystopian futures we've
discussed.
But it can't last forever, even if we
revert to medieval levels of technology.
We know from human history that it's
possible for humans [music] to climb out
of those conditions and invent modern
technology.
And there's another dark side to this.
You can't reach this world voluntarily.
Game [music] theory makes unilateral
disarmament impossible.
If one country gives up technology
[music] while others keep developing it.
The country that keeps developing wins.
They have the weapons. They have the
surveillance.
They have the AI.
You can't opt out unless everyone opts
out. If this world arrives, [music]
Techmark thinks it's more likely to
happen in a different way. For example,
a human- created pandemic kills everyone
who knows how to build science and
technology. And there's a violent tear
down of factories, cities, and all
infrastructure.
Because in a world with 8 billion
humans, there will always be holdouts.
Someone has to force the transition.
Someone has to kill the scientists.
Someone has to destroy the
infrastructure. There's no peaceful path
to an Amish world.
Which brings us to the final future.
What if we used technology to prevent
more technology?
Techmark compares this future to 1984.
Super intelligence is prevented not by
an AI, but a human-led Orwellian global
surveillance state. If you can't trust
AI to save us from AI, maybe the only
option is humans [music] watching humans
forever. The technology for this sort of
thing already exists. Current tech can
feasibly track every phone call, email,
search query, and [music] every credit
card transaction. Smartphones are
microphones and cameras in everyone's
pocket. Security cameras with facial
recognition are everywhere. And all of
this works with current machine
learning. No futuristic super
intelligent AI required. Prisons are
already using AI models to monitor phone
calls in prison to predict when crimes
are being contemplated, which is
technology straight out of Minority
Report. Imagine this, but not just for
prisoners. Every conversation and every
thought is monitored like this. Now
imagine this surveillance technology
supercharged with AI.
Billionaire Larry Ellison talks
excitedly about a vast AI fueled
surveillance system to ensure citizens
will be on their best behavior or as
Ival Harrari put it. Now for the first
time in history it is technically
possible to anhilate privacy. If you
think about authoritarian regimes
throughout history they always wanted to
monitor their citizens around the clock
but this was technically impossible.
Even in the Soviet Union, you know, you
have 200 million Soviet citizens. You
can't follow them uh all the time
because the the KGB didn't have 200
million agents. And even if the KGB
somehow got 200 million agents, that's
not enough because, you know, in in the
Soviet Union, it's still basically paper
bureaucracy. If a secret agent followed
you around 24 hours a day, at the end of
the day, they write a paper report about
you and send it to KGB headquarters in
Moscow. So imagine every day KGB
headquarters is flooded with 200 million
paper reports. Now to be useful for
anything, somebody needs to read and
analyze them. They can't do it. They
don't have the analysts. Therefore, even
in the Soviet Union, some level of
privacy was still the default. And
remember, you have two eyes and ears.
AIS will have billions of eyes and ears.
Every phone microphone and camera, every
webcam, [music] every surveillance
camera, you will be watched.
Could a less Orwellian version of this
work? Maybe we don't monitor everything,
but just monitor advanced AI. Some
people think so. [music] In their new
book, if anyone builds it, everyone
dies, why superhuman AI would kill us
all, researchers at the Machine
Intelligence Research Institute are
proposing that we monitor advanced AI
like we monitor nuclear weapons.
Reminder, we can do international
coordination like keep an eye on $100
million and up compute clusters without
creating a dystopian Orwellian global
surveillance totalitarian regime.
Actually, fun fact, you're already
living in a global surveillance regime
for loads of things like nukes and
bioweapons.
But when people say global surveillance
regime, they imply every aspect of life
is [music] surveiled, not keep an eye on
enriched uranium, a thing most people
agree is good. We should worry about
Orwellian dystopias, but countries
coordinate all the time on all kinds of
things. [music] And you're probably glad
they monitor stuff lunatics need to
create super Ebola. Safety is an [music]
achievement. Humanity didn't get here by
driving a car without brakes. We need
gas and brakes. We want to reach the
stars, too. But we'll never get there if
we go too fast and crash on the way. I
don't want powerful AI to be forbidden,
but it should be heavily regulated, not
something you can build in your
Midwestern garage or in North Korea.
international treaties with strict
enforcement and inspections as we do
with nukes. So if you build a rogue data
center, people will know consequences
for violations. We already did this with
nuclear weapons. We haven't eliminated
nukes, but we have prevented their
proliferation to just nine countries.
Back then, people thought this would be
impossible. Surely every country would
have nukes. Perhaps even thousands of
corporations, maybe even millions of
people. But no, just nine countries. And
nobody has died from a nuke in 80 years.
Could we do the same with AI? Maybe,
maybe not. Over time, AI will get
cheaper and cheaper to build. GPUs are
harder to control than enriched uranium,
but it would slow down AI. It would buy
us time to figure out what to do. Okay,
that's all 12. Let me know in the
comments which of these you think is
most likely. Which one we should steer
toward?
>> Alarm bells over the artificial
intelligence are [music] deafening and
they are loudest from the developers who
designed it.
The scientists and experts have called
on the world to act, declaring AI an
existential threat to humanity on a par
with the risk of nuclear war.
By the way, we don't get to not choose.
If we don't want that future, we have to
pick a better one. Otherwise, we could
wake up one day in a world where super
intelligent AI has taken over.
a scenario we go through step by step in
this video about how an AI escapes the
lab. Hey, I'm Drew and I really loved
making this one for you guys. Thank you.