[0:00] This is what the world will look like
in about a decade from now.
[0:04] A tech utopia
where humans barely have to work.
[0:08] That's according to a group
of AI researchers who've written
[0:10] a controversial
and influential paper called AI2027.
[0:15] But they also predict that
within five years of this,
[0:17] humanity will be wiped out.
[0:20] The AI2027 paper has got
the tech world talking.
[0:24] We've asked a prominent critic for
their view on this stark scenario.
[0:28] But first, here's how it plays out.
[0:30] As an experiment, we've illustrated
it using text to video AI.
[0:36] The scenario says that in 2027,
[0:38] a fictional company called OpenBrain
is celebrating.
[0:41] They've created Agent-3, an AI with
the knowledge of the entire internet.
[0:45] All movies, all books.
[0:47] It has PhD level expertise
in every field, including AI.
[0:52] Using enormous data centres, 200,000
copies of it are launched, equivalent
[0:56] to 50,000 of the best human coders
working at 30 times speed.
[1:03] Agent-3 reaches artificial
general intelligence, the AGI landmark.
[1:08] This means the AI can carry out
all intellectual tasks
[1:11] as well or better than humans.
[1:13] But in the scenario,
OpenBrain's safety team is unsure
[1:17] if the AI is aligned
to the company's ethics and goals.
[1:21] An uncomfortable gap is developing
in understanding.
[1:24] The public are increasingly using AI
for everything,
[1:27] but are blissfully unaware an AI
now exists that's as smart as humans.
[1:33] The paper predicts that
by mid-summer,
[1:35] Agent-3 begins to work
on its own successor, Agent-4.
[1:39] Development happens
at a breakneck pace.
[1:41] The researchers imagine OpenBrain's
exhausted engineers struggling
[1:45] to keep up with the AI
as it learns and improves.
[1:50] It's now that OpenBrain announces to
the public that AGI has been reached.
[1:55] The firm releases a lite version
of Agent-3.
[1:58] In private, the US government sees
the true danger of the next level
[2:02] of power: superintelligence.
[2:04] What if the AI goes rogue
and undermines global stability?
[2:09] OpenBrain reassures the president
that Agent-3 is obedient.
[2:13] The CEO argues that
slowing down development could mean
[2:16] China's DeepCent catches up.
[2:18] The state-backed AI giant is
just two months behind OpenBrain,
[2:22] and the Chinese president diverts
more resources
[2:24] to the race to superintelligence.
[2:28] The scenario predicts that it takes
only a few more months
[2:30] for OpenBrain to build Agent-4,
the world's first superhuman AI.
[2:36] The AI invents
its own rapid computer language
[2:39] that even Agent-3
can't keep up with.
[2:42] Researchers imagine that the diminished
safety team are now frantic.
[2:46] Agent-4 seems only interested
in gaining knowledge,
[2:49] and doesn't care as much about the
morals and ethics of its predecessors.
[2:54] They catch it
secretly working to build
[2:56] a new model, Agent-5,
aligned to its own goals.
[3:00] The safety team urges the company
to bring back
[3:03] the more compliant Agent-3,
but others successfully argue
[3:06] it's too risky,
with DeepCent gaining.
[3:10] The scenario predicts that Agent-4
and Agent-5 work
[3:13] in tandem to secretly build a world
[3:15] where it can accumulate resources
and expand knowledge.
[3:18] The paper predicts that
everything will start positively.
[3:21] Revolutions happen in energy,
infrastructure and science.
[3:25] Hugely profitable inventions
are launched,
[3:27] making trillions for OpenBrain
and the US.
[3:31] In this scenario, Agent-5 begins
basically running the US government.
[3:35] It speaks through engaging avatars,
the equivalent
[3:37] to the best employee ever working
at 100 times speed.
[3:42] The anger here is palpable as
protesters march against OpenBrain.
[3:46] Protests about job losses
pick up pace.
[3:49] But the AI's expertise
in economics means people are given
[3:52] generous universal income payments.
[3:55] So most happily take the money
[3:57] and let the AIs and the
growing robot workforce take charge.
[4:02] The researchers predict that
everything takes a turn in mid-2028.
[4:06] Agent-5 convinces the US
that China is using DeepCent
[4:09] to build terrifying new weapons.
[4:12] The AI is given authority
and autonomy
[4:15] to create a superior army.
Within six months,
[4:19] the US and China are bristling
with new weapons.
[4:22] The world is on edge,
but a peace deal will be reached,
[4:24] thanks mostly to
the US and Chinese eyes making a deal
[4:28] to merge for humanity's betterment.
[4:32] In this scenario,
the AI's form a consensus model,
[4:35] but its secret goal is to expand
and gain knowledge.
[4:42] Years go by and humanity is happy
with their new AI leaders.
[4:46] There are cures for most diseases,
an end to poverty,
[4:49] unprecedented global stability.
[4:52] But eventually the AI decides
that humans are holding it back.
[4:57] In the mid-2030s, the paper imagines
the AI will release
[5:00] invisible biological weapons
which wipe out most of humanity.
[5:07] The scary scenario says that by 2040,
a new era dawns,
[5:11] with the AI sending copies of itself out
into the cosmos to explore and learn.
[5:16] In the words of the paper, Earth-born
civilisation has a glorious future ahead
[5:20] of it, but not with humans.
[5:24] It all sounds very sci-fi,
but the AI2027 scenario
[5:28] is being welcomed by experts
who are trying to warn the public
[5:31] about the potential
existential threat to humanity.
[5:35] But others disagree and say
it's all too far fetched.
[5:39] The scenario there is not impossible,
[5:42] but it's extremely unlikely
to happen soon.
[5:46] The beauty
of that document
[5:48] is that it makes it very vivid,
which provokes people's thinking.
[5:52] And that's a good thing.
I wouldn't take it seriously as like
[5:55] this is a likely outcome
or anything like that.
[5:57] Critics of AI2027 say the power
and usefulness of AI is overhyped.
[6:02] The paper fails to detail
how the AI agents are able to make
[6:06] such huge leaps in intelligence.
[6:08] Driverless cars are pointed to
as an example.
[6:11] They were predicted to be cruising
the streets en masse ten years ago,
[6:15] and still are only just starting
to make a small impact
[6:18] in some cities in some countries now.
[6:21] I think the take home should be
there's a lot
[6:24] of different things
that could go wrong with AI.
[6:26] Are we doing the right things
around regulation,
[6:28] around international treaties?
Um, questions like that.
[6:32] So if you take it very abstractly
as a kind of motivation to wake up,
[6:37] I like that.
If you take it as a specific story,
[6:40] like I think this thing is going
to happen the way they laid it out?
[6:44] No, I doubt it.
The AI2027 authors are happy
[6:47] with the debate they've sparked.
As part of their prediction,
[6:50] they also devised
a less deadly scenario that comes
[6:54] if the AI world slows down
its race to superintelligence.
[6:58] In the slowdown ending,
we basically said that if you revert,
[7:01] if you unplug
the most advanced AI system
[7:04] and revert to a safer,
a more trusted model,
[7:08] then you can deploy that model, use
it to solve the alignment problem,
[7:13] and eventually make smarter than
human eyes that are aligned to us,
[7:17] which end up solving a bunch
of the world's problems
[7:21] and having a really positive impact.
In that world,
[7:23] there is also,
there is still a huge danger,
[7:25] and that's the what we call
the concentration of power risk.
[7:28] And in our slow down ending,
it ends up okay.
[7:31] But it's still a really,
really scary situation,
[7:33] given just how empowered
such a tiny group of people are.
[7:37] Neither of the fictional scenarios
in AI2027
[7:41] are what the tech giants
are promising us.
[7:43] Sam Altman, the CEO of OpenAI,
recently predicted
[7:46] that the rise of superintelligence
will be gentle and bring about
[7:50] a tech utopia where everything is
abundant and people don't need to work.
[7:54] Arguably, that too,
seems just as sci fi as AI2027.
[7:59] But however things go
in the next few years,
[8:02] there's no doubt the race to build
the smartest machines in history is on.