AI2027: Is this how AI might destroy humanity? - BBC World Service

[00:00] This is what the world will look like

[00:04] A tech utopia

[00:08] That's according to a group

[00:10] a controversial

[00:15] But they also predict that

[00:17] humanity will be wiped out.

[00:20] The AI2027 paper has got

[00:24] We've asked a prominent critic for

[00:28] But first, here's how it plays out.

[00:30] As an experiment, we've illustrated

[00:36] The scenario says that in 2027,

[00:38] a fictional company called OpenBrain

[00:41] They've created Agent-3, an AI with

[00:45] All movies, all books.

[00:47] It has PhD level expertise

[00:52] Using enormous data centres, 200,000

[00:56] to 50,000 of the best human coders

[01:03] Agent-3 reaches artificial

[01:08] This means the AI can carry out

[01:11] as well or better than humans.

[01:13] But in the scenario,

[01:17] if the AI is aligned

[01:21] An uncomfortable gap is developing

[01:24] The public are increasingly using AI

[01:27] but are blissfully unaware an AI

[01:33] The paper predicts that

[01:35] Agent-3 begins to work

[01:39] Development happens

[01:41] The researchers imagine OpenBrain's

[01:45] to keep up with the AI

[01:50] It's now that OpenBrain announces to

[01:55] The firm releases a lite version

[01:58] In private, the US government sees

[02:02] of power: superintelligence.

[02:04] What if the AI goes rogue

[02:09] OpenBrain reassures the president

[02:13] The CEO argues that

[02:16] China's DeepCent catches up.

[02:18] The state-backed AI giant is

[02:22] and the Chinese president diverts

[02:24] to the race to superintelligence.

[02:28] The scenario predicts that it takes

[02:30] for OpenBrain to build Agent-4,

[02:36] The AI invents

[02:39] that even Agent-3

[02:42] Researchers imagine that the diminished

[02:46] Agent-4 seems only interested

[02:49] and doesn't care as much about the

[02:54] They catch it

[02:56] a new model, Agent-5,

[03:00] The safety team urges the company

[03:03] the more compliant Agent-3,

[03:06] it's too risky,

[03:10] The scenario predicts that Agent-4

[03:13] in tandem to secretly build a world

[03:15] where it can accumulate resources

[03:18] The paper predicts that

[03:21] Revolutions happen in energy,

[03:25] Hugely profitable inventions

[03:27] making trillions for OpenBrain

[03:31] In this scenario, Agent-5 begins

[03:35] It speaks through engaging avatars,

[03:37] to the best employee ever working

[03:42] The anger here is palpable as

[03:46] Protests about job losses

[03:49] But the AI's expertise

[03:52] generous universal income payments.

[03:55] So most happily take the money

[03:57] and let the AIs and the

[04:02] The researchers predict that

[04:06] Agent-5 convinces the US

[04:09] to build terrifying new weapons.

[04:12] The AI is given authority

[04:15] to create a superior army.

[04:19] the US and China are bristling

[04:22] The world is on edge,

[04:24] thanks mostly to

[04:28] to merge for humanity's betterment.

[04:32] In this scenario,

[04:35] but its secret goal is to expand

[04:42] Years go by and humanity is happy

[04:46] There are cures for most diseases,

[04:49] unprecedented global stability.

[04:52] But eventually the AI decides

[04:57] In the mid-2030s, the paper imagines

[05:00] invisible biological weapons

[05:07] The scary scenario says that by 2040,

[05:11] with the AI sending copies of itself out

[05:16] In the words of the paper, Earth-born

[05:20] of it, but not with humans.

[05:24] It all sounds very sci-fi,

[05:28] is being welcomed by experts

[05:31] about the potential

[05:35] But others disagree and say

[05:39] The scenario there is not impossible,

[05:42] but it's extremely unlikely

[05:46] The beauty

[05:48] is that it makes it very vivid,

[05:52] And that's a good thing.

[05:55] this is a likely outcome

[05:57] Critics of AI2027 say the power

[06:02] The paper fails to detail

[06:06] such huge leaps in intelligence.

[06:08] Driverless cars are pointed to

[06:11] They were predicted to be cruising

[06:15] and still are only just starting

[06:18] in some cities in some countries now.

[06:21] I think the take home should be

[06:24] of different things

[06:26] Are we doing the right things

[06:28] around international treaties?

[06:32] So if you take it very abstractly

[06:37] I like that.

[06:40] like I think this thing is going

[06:44] No, I doubt it.

[06:47] with the debate they've sparked.

[06:50] they also devised

[06:54] if the AI world slows down

[06:58] In the slowdown ending,

[07:01] if you unplug

[07:04] and revert to a safer,

[07:08] then you can deploy that model, use

[07:13] and eventually make smarter than

[07:17] which end up solving a bunch

[07:21] and having a really positive impact.

[07:23] there is also,

[07:25] and that's the what we call

[07:28] And in our slow down ending,

[07:31] But it's still a really,

[07:33] given just how empowered

[07:37] Neither of the fictional scenarios

[07:41] are what the tech giants

[07:43] Sam Altman, the CEO of OpenAI,

[07:46] that the rise of superintelligence

[07:50] a tech utopia where everything is

[07:54] Arguably, that too,

[07:59] But however things go

[08:02] there's no doubt the race to build