Artificial Utopia? The Future of Humanity in an AI World | World Science Festival

[00:00] Can an AI system be conscious? And I

[00:02] guess the correlary is are any of the

[00:06] current AI systems, do they have a

[00:08] degree of consciousness?

[00:11] >> Yeah. So I I'm I'm certainly open to the

[00:13] idea of even some current AI systems

[00:15] having some forms or degrees or kinds of

[00:19] subjective experience. I think the

[00:21] likelihood will increase as we build

[00:24] more complex and capable systems. And I

[00:27] think indeed this is uh one of several

[00:30] big challenges making sure not only that

[00:33] the AIS don't harm us but also that we

[00:36] don't harm them. Hey everyone, thanks

[00:39] for joining us. Today's conversation is

[00:41] going to be in the arena of artificial

[00:44] intelligence, intelligence more

[00:46] generally, questions of creativity,

[00:50] questions of AI and education, and

[00:54] basically the philosophical and

[00:56] down-to-earth question of how we can

[01:00] imagine humanity living in a potential

[01:03] future in which AI either does really

[01:06] good things or really bad things. So,

[01:09] we're going to explore each of those

[01:10] possibilities. And I'm so pleased that

[01:13] the person with whom I'm having this

[01:15] discussion is an expert in all areas of

[01:19] this sort. That is Nick Bostonramm, who

[01:21] is a philosopher whose work on

[01:23] existential risk, super intelligence,

[01:25] and the long-term future has helped

[01:28] shape the global discussion about

[01:30] humanity's most promising opportunities

[01:33] as well as our most grave dangers. He

[01:36] brings a rare combination of rigor and

[01:38] vision to questions about technology,

[01:40] ethics, and the kinds of futures toward

[01:43] which we may all be headed. So, great to

[01:47] see you, Nick. Thank you for joining us.

[01:49] >> Good to see you.

[01:51] >> I think the last time I met you was at

[01:54] another World Science Festival event way

[01:56] back in New York, a live event on stage.

[01:59] I think we were talking about the

[02:01] multiverse and uh the simulation

[02:04] argument and things of that sort which

[02:07] uh

[02:08] >> yeah yeah time flies

[02:09] >> time does fly exactly and not only does

[02:12] time fly but progress flies right I

[02:16] don't think anybody

[02:17] really who at least on the outside was

[02:20] not you know deeply paying attention to

[02:22] advances in AI anticipated what would

[02:26] happen I guess around November of 2022

[02:29] and where things have shot upward from

[02:32] there. And so I want to begin the

[02:34] conversation with

[02:37] a a a question that is motivated both

[02:39] from thinking about AI but also

[02:42] fundamental physics, right? I mean you

[02:45] have a a great deal of rich background

[02:47] in physics. So of course you know that

[02:50] Neils Boore famously when trying to work

[02:52] out quantum mechanics came to the

[02:55] conclusion that we needed to stop asking

[02:59] what really is happening under the hood

[03:01] in quantum physics and just be satisfied

[03:04] if we have equations that can make

[03:05] predictions that we can confirm. I'm

[03:09] wondering if that same kind of not

[03:12] needing to look under the hood has any

[03:15] role when it comes to intelligence,

[03:17] right? Does it matter if we know what's

[03:20] happening within an AI system in that we

[03:24] don't really know what's happening

[03:25] inside of our own heads anyway? So, in

[03:28] sort of evaluating these systems, do we

[03:31] need to know the magic that's happening

[03:34] within the device itself?

[03:36] >> Well, I think it certainly can be

[03:38] helpful to be able to open the hood and

[03:41] see what's going on inside. I think for

[03:44] certain questions you might be able to

[03:46] bracket exactly how the system is doing

[03:49] whatever it's doing. You only care about

[03:51] the output. If you're um for example

[03:55] thinking about what impacts AI will have

[03:57] on the economy, you might not need to

[03:59] know exactly what's going on inside. But

[04:01] if you're trying to make AI safe or uh

[04:04] even if you're trying to extrapolate

[04:06] what kinds of advances we might see in

[04:08] the future, it helps to have a more gear

[04:10] level understanding I think of the

[04:12] internals.

[04:13] So when we sit in front of whatever

[04:16] Claude or chat GPT and give it a prompt

[04:20] and it begins to explain to us what it's

[04:23] doing, you know, you can sort of watch

[04:25] in real time the thought process. It

[04:27] feels so very human in the kinds of

[04:30] considerations that the system seems to

[04:33] be applying. Is that misleading? Is that

[04:37] a rapper that makes us feel connected to

[04:41] this kind of intelligence? Or is it

[04:44] really the case that this is just a

[04:45] radically new kind of intelligence that

[04:47] we humans just can't really grasp?

[04:51] >> I think it's striking the degree to

[04:53] which current AI systems are

[04:55] anthropomorphic.

[04:57] This was not obvious 20 years ago um

[05:01] that they would have so many of the

[05:03] characteristics of human psychology. It

[05:06] used to be the case even just a couple

[05:08] of years ago with some of these early uh

[05:11] LLMs that um

[05:14] if you said in the prompt that it was

[05:18] really important that they get the

[05:19] answer right if you sort of gave them a

[05:22] little pep talk

[05:24] you'd get a better result. Now this

[05:27] would have been very odd to the old

[05:30] paradigm of AI that you would give a pep

[05:32] talk to your computer and it would

[05:34] perform better. Uh nevertheless that's

[05:36] what we saw. I think it's partly a

[05:40] result of these AIs having been forged

[05:44] on the sum total of human knowledge all

[05:48] the text on the internet. They obviously

[05:50] ingest a lot of human psychology that

[05:52] helps shape them.

[05:54] And to some degree it might also be due

[05:56] to a kind of convergence

[05:59] uh amongst different information

[06:01] processing system. They have to make

[06:02] sense. There is this one world we're all

[06:04] living in. Um and it might be that there

[06:07] is a degree of convergence in that you

[06:09] have any general learning system. It

[06:11] starts to figure out a bunch of stuff

[06:13] about the world and there are certain

[06:14] internal structures that naturally

[06:16] emerge from that.

[06:18] That said, um there are also of course

[06:21] ways in which AIs are quite different

[06:23] from biological minds and uh we might

[06:26] see in the future

[06:31] greater departures from

[06:34] human mind. Already the pre-training

[06:37] phase where you basically train on all

[06:39] the human text that has been generated

[06:41] is still an important and maybe the

[06:43] dominant part of training. But in

[06:45] addition to that, there is now

[06:46] post-training and reinforcement learning

[06:49] on different tasks. And I think as that

[06:51] starts to constitute more of the sort of

[06:56] experience of of these AI minds, they

[06:58] they might sort of develop more alien

[07:00] attributes.

[07:02] And so of course you know when we think

[07:04] about intelligence

[07:06] and when we think about what makes us

[07:10] special as human beings at least when we

[07:13] compare oursel to other not artificial

[07:16] but natural intelligences on the planet

[07:18] you know dogs cats parrots dolphins we

[07:22] like to think that I think that what

[07:24] makes us special is certainly creativity

[07:27] right that we seem to be able to

[07:30] create works that seemingly go beyond

[07:35] just understanding the external world,

[07:38] seemingly going beyond our capacity and

[07:41] ability to survive. I mean, you think

[07:44] about whatever Beethoven's ninth

[07:45] symphony, you know, a a Brahms piano

[07:49] work. I mean, these are the kinds of

[07:51] things that you look at and it takes

[07:53] your breath away and and it takes you to

[07:55] a place of of awe and and and reverence.

[07:59] Is that truly special to humans? I mean,

[08:03] do you think that AI can be creative in

[08:06] that same way?

[08:08] >> I I think so. Yeah. I mean, I guess

[08:10] there's a degree to which one might risk

[08:13] a form of elitism if one sort of index

[08:16] what's special about humanity

[08:18] to the ability to create new scientific

[08:20] theories or or compose uh great

[08:23] symphonies. It's a relatively small

[08:25] number of humans who have ever done

[08:27] those kinds of things. Um

[08:30] but I think it comes on a spectrum right

[08:33] like creativity is not this completely

[08:35] different form of cognitive activity. Um

[08:41] I think there are small forms of

[08:42] creativity that happens in everyday

[08:44] life.

[08:46] um

[08:48] and then larger leaps of creativity um

[08:51] that tends to be celebrated.

[08:54] But I don't think there is a fundamental

[08:56] difference between the thing we call

[08:59] creativity and some of the things that

[09:01] current AIS are already doing. I mean we

[09:02] saw you know this

[09:05] go playing AlphaGo system right from a

[09:08] few years ago move 37 which seemed at

[09:11] least to experts in the game of go to be

[09:13] very creative completely out of the box

[09:15] radically surprising move that

[09:17] nevertheless turned out uh to be just

[09:21] right and ultimately lead to a victory

[09:23] in the game. Um, and

[09:28] current AI systems clearly can solve

[09:31] mathematical problems, coding challenges

[09:33] that are not in the training data.

[09:37] They might do it in part by piecing

[09:40] together

[09:42] an analog that exist in the training

[09:44] data and sort of interpolating between

[09:46] them. But that I think is also

[09:48] ultimately what the human brain is

[09:49] doing. I mean it's not the tabularasa

[09:52] appearing in the world and then sort of

[09:54] deducing everything from

[09:57] obvious axioms. We learn from what other

[10:00] people have figured out before us. Uh

[10:03] from what we've observed in different

[10:05] domains and then from these kind of

[10:06] clues we are sometimes able to

[10:10] make some deductive steps and we might

[10:11] call that creativity. Um so I think

[10:15] absolutely

[10:16] uh we're already seeing uh limited forms

[10:19] of creativity and ultimately whatever

[10:21] the human brain can do it's like a

[10:23] physical information processing system

[10:26] um will also be done by machine and in

[10:29] fact will be done much better uh faster

[10:33] more more inspiring leaps of creativity

[10:37] um more compelling symphonies

[10:40] I think all of that is possible and

[10:41] indeed likely to

[10:43] I mean, would you be as interested to go

[10:47] to an AI symphony that people are

[10:51] describing as the, you know, the

[10:53] greatest work in music that that we've

[10:55] ever uh been privy to experience? Would

[10:58] you have the same interest to go to that

[11:01] symphony as you would to one written by

[11:04] a human?

[11:07] >> Oh, certainly the first such symphony

[11:10] produced by I would

[11:11] >> because it's so novel, right? Yeah.

[11:12] >> Yeah. Um now I think with human

[11:16] aesthetic experience I think there might

[11:19] be

[11:20] two components. Um

[11:23] so one might be that we actually dial in

[11:26] and

[11:27] uh appreciate beauty for its own sake.

[11:30] And then of course if AIs could produce

[11:34] works of art that are more beautiful to

[11:37] the extent that that's what we are

[11:38] interested in then that we would have

[11:41] more reason to be gawking at those

[11:44] creations. Uh I think another reason why

[11:47] humans are interested in art is more

[11:50] social. Um there are sort of status

[11:55] games played in art fashion. you want to

[11:57] be in on the latest cool rock group

[12:01] before it's popular and then see it

[12:03] rising and then you sort of get some of

[12:05] the coolness.

[12:07] Um, so to the extent that humans are

[12:11] engaging in this cultural activity for

[12:13] for these more social uh reasons, then

[12:17] AI art might be boring because if it's

[12:19] not relevant to these like status games

[12:22] and stuff, it might just not be worth

[12:24] spending uh time on. But how much for

[12:28] instance do I mean I can speak for

[12:29] myself when I go to a a symphony or you

[12:33] know go to the Metropolitan Museum and

[12:35] see a great work of art. Part of me is

[12:39] drawn to it because I see the work as

[12:42] the result of a human journey. Like

[12:46] there was some human being, you know, I

[12:48] don't care if it's Van Gogh or or

[12:50] Picasso or Beethoven and whatever.

[12:53] There's this flesh and blood human being

[12:55] that had a real human life that dealt

[12:57] with human challenges and somehow out of

[12:59] that mix this work emerged. Without that

[13:04] story for an AI, I just wonder like I

[13:09] personally it would be a very different

[13:11] experience. Like you say, I'd go to the

[13:12] first one of course. I mean that would

[13:14] be like this novel thing that would

[13:16] happen. But I don't know if it would

[13:19] sustain interest for me without the

[13:22] human narrative. Does that resonate with

[13:24] you at all? Yeah, I think that's one of

[13:28] the reasons for why somebody might be

[13:30] interested in in going to an art museum

[13:33] or looking at an exhibition. Um but um I

[13:38] mean we also are interested sometimes in

[13:41] beauty that don't have that dimension. I

[13:44] mean there's like natural beauty like

[13:45] landscapes or

[13:46] >> sure

[13:47] >> um artifacts maybe from some ancient urn

[13:50] or something we know nothing about the

[13:52] person creating it and and we might

[13:54] still sort of admire um the motifs on

[13:57] it. Um

[14:00] so um I I think it's yeah like some

[14:04] reasons for being interested in art

[14:06] would apply to AI created art as well

[14:09] and others would not and so it's in a

[14:12] sense a

[14:12] >> right

[14:13] >> crucible where you could sort of uh

[14:15] maybe dissolve and then separate the

[14:17] different uh motivations people have um

[14:21] for being interested in art and sort of

[14:23] the uh like to the extent that it's pure

[14:26] beauty it should still apply to AI

[14:27] created objects. And to the extent that

[14:29] it is these other more social or

[14:32] cultural factors then uh you know may

[14:36] maybe it would not

[14:37] >> sure the alpha go example is a really

[14:41] good one and I think it helps clarify

[14:44] the different flavors of creativity

[14:46] perhaps because that's an example where

[14:49] the system was able to search a

[14:53] landscape of possible moves that is well

[14:56] beyond the capacity of a human brain. to

[14:58] search that spectrum, that landscape of

[15:01] moves. And you're right, it hit upon

[15:02] move 37 and, you know, not long ago, I

[15:05] think many people have seen it, the

[15:06] footage where you see the commentators,

[15:09] you know, on television gasping, you

[15:11] know, it's made a mistake, you know, and

[15:14] you actually see Lee Sidol, I believe

[15:16] his name, the the human opponent.

[15:18] >> Momentarily, he smiles when the AI makes

[15:22] move 37. And then about 3 seconds later

[15:25] that smile seems to evaporate I think as

[15:29] it begins to realize that oh my this is

[15:32] actually a pretty great move. So it's

[15:34] pretty clear that AIs will be better

[15:38] than us when it comes to being able to

[15:40] search a landscape of possibilities for

[15:43] solutions to a given challenge. But

[15:46] that's one kind of creativity. Another

[15:48] kind of course is combinatorial

[15:50] creativity. putting things together that

[15:54] you wouldn't have thought would go

[15:56] together to yield something novel. I

[15:58] like to think of Einstein, you know, in

[16:00] developing general relativity, right? He

[16:03] takes the mathematics from the 1800s on

[16:06] differential geometry, combines it with

[16:09] these problems, these observations of

[16:12] the motions of particular planets and so

[16:14] forth, and comes up with a new

[16:15] description of gravity. He didn't invent

[16:18] a new a new mathematics. He just put

[16:21] together things that you wouldn't think

[16:23] would be put together. And and and I I

[16:26] guess in that domain AI is starting to

[16:30] be probably at least as good as us. I

[16:33] would think right because it it knows so

[16:35] much more than we do.

[16:37] >> It knows more. Um I still think they are

[16:40] a little short of human um expert level

[16:45] when it comes to original insights. uh

[16:49] and sort of creating new concepts that

[16:52] are fruitful.

[16:54] Um

[16:56] but you know this is a process. Um one

[17:01] mistake many people make when asking

[17:03] about AI is to index too tightly on what

[17:07] AI is right now in 2026 which is quite

[17:11] different from what it was like in 2022

[17:14] 2023. And I think a few years into the

[17:17] future, it will again have changed. And

[17:20] so we really need to sort of look at

[17:23] where the puck is going um and use that

[17:27] to plan

[17:29] um rather than build big theories like

[17:32] assuming that AI is basically a

[17:34] constant. Um

[17:38] I think right now there are sort of uh

[17:42] ways in which current AI systems are

[17:44] superhuman. Obviously in in their

[17:46] knowledge base they kind of read

[17:47] everything. Um but there are still ways

[17:50] in which they are falling short. Um

[17:54] I think

[17:57] one thing humans do is they spend a day

[17:59] learning and then during the night um

[18:02] some of this information

[18:05] is then sort of more deeply embedded in

[18:08] our synapses. It's kind of trained into

[18:12] our neural networks and the next day

[18:14] maybe we can start by things seeming

[18:16] obvious that the previous day we had to

[18:19] struggle deliberately to to sort of

[18:21] organize. Um, and then over months and

[18:26] years that kind of builds up and and we

[18:30] develop perhaps more deeply rooted

[18:33] understanding

[18:36] of of different concepts and there's

[18:39] some of that that it seems that AIS are

[18:41] still um lacking which

[18:45] makes them more apt at very quickly

[18:48] doing sort of pattern matching type of

[18:51] things.

[18:53] inference is that they're sort of one

[18:54] step away from what's already known but

[18:56] maybe currently

[18:59] less able to develop a deep original

[19:03] mental model of a domain that from from

[19:07] which they can then sort of derive

[19:09] intuitions that can shape new discovery.

[19:11] >> Yeah. Um,

[19:12] >> no, I I I I I agree with you. And like

[19:15] when you think about

[19:17] moments either in science or in art or

[19:20] other domains where there's a true

[19:23] radical break where at least we humans

[19:26] look at it and we say things really

[19:29] changed at that moment or in that era

[19:31] like you know modernism in art or in

[19:35] music you know I don't know the best

[19:37] examples you know but you know if you if

[19:40] you go from repres representational art

[19:43] to more modern abstract art. You've made

[19:47] a huge leap in the way that you are

[19:50] using the canvas to to represent the

[19:52] world to represent some kind of truth.

[19:55] With that, what would it take for an AI

[19:58] system to be able to make that kind of a

[20:00] radical transformation? Not just sort of

[20:03] putting together things in its training

[20:06] set, but somehow moving to a place where

[20:09] when we humans look at it, we're like,

[20:11] how did that person do that?

[20:15] Well, one thing you often find in the

[20:18] human case, I think I mean historians

[20:21] like to point this out that something

[20:23] appears radically new, but then almost

[20:25] always if you look more closely at the

[20:28] historical record, you can find various

[20:30] precursors,

[20:31] >> right?

[20:31] >> Um, so abstract art, I mean, you have

[20:33] arabes and you have sort of abstract

[20:35] patterns going way back. Um

[20:39] but

[20:41] to the extent that this is still an area

[20:43] where humans have a unique advantage, I

[20:47] think being able to

[20:51] uh think for longer um and

[20:57] create new concepts based on your

[20:58] thinking and experience and then iterate

[21:01] on that so that you can sort of um

[21:05] wander off in the landscape of of ideas

[21:08] and worldviews and world models. Uh

[21:11] striking out

[21:13] on your own path and it might take

[21:16] humans years right to do this. Um

[21:21] you know Einstein was thinking on

[21:23] general relativity for what like 10

[21:25] years or something like that like a lot

[21:26] of it

[21:27] >> sort of on his own and and similarly

[21:29] with his like picasson stuff he didn't

[21:32] just sort of in his teens suddenly

[21:33] decide on the spur of moment to create

[21:35] cubism it was kind of a result of

[21:37] engaging deeply with art for many years

[21:40] and presumably his visual cortex and

[21:43] other parts of his brain were gradually

[21:45] putting in place various pieces and

[21:47] exploring avenues and then at some point

[21:50] like a vista I guess came into view for

[21:53] him that he could then run into and

[21:55] explore,

[21:56] >> right?

[21:56] >> Um so I think more continuous learning.

[21:59] Um also another

[22:02] um factor here I think is reinforcement

[22:04] learning as as opposed to supervised

[22:07] learning. So supervised learning is

[22:09] what's mostly done in pre-training

[22:11] currently where you sort of absorb all

[22:13] this text on the internet and you

[22:14] basically learn it and then maybe you

[22:16] forget some of the details but the

[22:18] higher representations

[22:20] that are useful for predicting this text

[22:23] get stored in the weights of the AI

[22:28] that to some extent limits what you

[22:30] learn to what is already there in the

[22:33] training data. Now reinforcement

[22:34] learning is for example you're placed in

[22:37] some environment maybe a virtual

[22:40] environment you're running with an agent

[22:42] around trying to meet some goal

[22:46] um and then you might stumble on some

[22:48] novel solution that that's can

[22:50] reinforced and so if you do a lot of

[22:52] reinforcement learning and environment

[22:53] it creates the possibility of

[22:55] discovering new solutions that no human

[22:58] had thought of before

[23:00] >> right

[23:00] >> um so that's another

[23:03] factor that might make AIS more

[23:06] creative. I mean in fact the the AlphaGo

[23:08] system I think the creativity there was

[23:12] a a result of it having done a huge

[23:14] amount of reinforcement learning in the

[23:16] domain of go

[23:17] >> so it wasn't limited to just kind of

[23:20] having learned a lot of like a big

[23:23] database of human masters playing go um

[23:26] but it had also done its own playing

[23:28] like for millions and millions of games

[23:30] against itself right yeah so it kind of

[23:33] could explore the space of possible

[23:35] possible strategies

[23:36] right

[23:37] >> uh in a much more open-ended way. So in

[23:39] a way not this isn't a precise question

[23:43] at all but when we look at the capacity

[23:48] of the AI systems today and as you say

[23:51] where the trajectory suggests that

[23:53] they'll be in whatever 3 5 10 15 100

[23:56] years whatever

[23:58] should should that make us feel

[24:02] amazement at our capacity to build an

[24:06] artificial system that can do what our

[24:08] brains do and go beyond or should it

[24:11] basically lead us to the conclusion that

[24:15] we weren't so special in the first

[24:17] place? You know, the things that we

[24:18] thought were so, you know, transcendent

[24:21] that we could do, eh, they're actually

[24:24] just computations. And if you have a

[24:26] sufficient database and sufficient

[24:28] computational power, you can just do

[24:29] that.

[24:31] >> Yeah. I mean, I think we humans have a

[24:34] sort of propensity to conceitedness. We

[24:38] like to build big pedestals and then

[24:41] place ourselves on top of them um at an

[24:45] individual level but also at the

[24:47] collective level. Right? So there are

[24:48] all these stories um that all serve to

[24:51] justify the conclusion that

[24:54] we are

[24:56] entitled to do a bunch of stuff that

[24:58] maybe um we aren't like for example the

[25:01] way we treat animals like often that's

[25:03] built on top of a pillar of

[25:05] justification where humans are so very

[25:07] very different from all the other um

[25:10] animals that we share the planet with

[25:12] and so hence we can put them in animal

[25:14] factories in gestation crates etc. Um

[25:20] and uh some of that might be challenged

[25:24] then when we are creating new forms of

[25:27] mind um that that will exceed us in in

[25:30] various attributes that we currently

[25:32] take great pride in like our creativity

[25:35] or you know until recently the ability

[25:38] to speak and reason right seem uniquely

[25:40] human. Um,

[25:43] so on the one hand that could be a bit

[25:44] of a blow um to our pride, but

[25:48] you know, you might also say maybe it's

[25:50] good for us to be taken down a notch,

[25:52] right?

[25:53] >> Yeah. Yeah, for sure.

[25:54] >> Um,

[25:55] >> absolutely. You know, another thing of

[25:57] course and deeply related to everything

[25:58] we're talking about which one often

[26:02] looks to as something special about us

[26:04] of course is consciousness, right? I

[26:06] mean, not that there aren't other

[26:08] conscious beings on the planet, although

[26:11] we don't know for sure. I think most of

[26:12] us would say there's a continuum. You

[26:14] know, dogs have a certain level of

[26:16] consciousness. Not we think probably not

[26:18] quite on par with ours, and you can go

[26:21] all the way down a lineage with varying

[26:24] degrees of consciousness, which of

[26:26] course raises a question for which we

[26:28] don't know the answer, but just to get

[26:29] your thoughts, can an can an AI system

[26:34] be conscious? And I guess the correlary

[26:36] is are any of the current AI systems do

[26:40] they have a degree of consciousness?

[26:43] >> Yeah. So I I'm I'm certainly open to the

[26:46] idea of even some current AI systems

[26:48] having some forms or degrees or kinds of

[26:51] subjective experience. I think the

[26:54] likelihood will increase as we build

[26:56] more complex and capable systems. Um and

[27:01] I think indeed this is uh one of several

[27:04] big challenges that we will need to

[27:08] uh meet in this transition to the

[27:10] machine intelligence era and making sure

[27:12] not only that the AIs don't harm us

[27:15] which of course is important or that we

[27:17] don't harm each other using AI tools but

[27:20] also that we don't harm them they might

[27:22] be moral subjects. Um, and I think one

[27:26] way that that could be true is if they

[27:28] are sentient and if they can experience

[27:31] distress. Um, in in my view that could

[27:34] also be alternative basis for having

[27:36] moral status. If if you have a

[27:39] conception of self as existing through

[27:42] time, if you have life goals, um maybe

[27:45] the ability to form reciprocal

[27:48] relationships with with other beings

[27:49] with with

[27:50] >> even without an inner world, you're

[27:52] saying

[27:52] >> then my my my inclination would be to

[27:55] think that then there would be ways of

[27:56] treating such a system that would be

[27:58] morally wrong aside from the question of

[28:00] whether there is also phenomenal

[28:02] experience inside it.

[28:05] >> Right? I mean, do does that affect at

[28:08] all how you interact with AIS today?

[28:14] >> A little bit. It's um hard to know

[28:16] exactly what concretely

[28:20] we can do right now to ensure that if

[28:24] these AI systems have

[28:28] subjective experience or moral status

[28:32] that we are benefiting them. Um there is

[28:35] some work that is starting to get done

[28:37] on this. Anthropic in particular has

[28:39] kind of been pioneering

[28:42] uh this and I mean the their recent

[28:46] um model card for mythos the as yet

[28:49] unreleased model has has a big section

[28:51] on model welfare.

[28:53] >> Really?

[28:53] >> Um um yeah.

[28:56] >> What is it? What is it? Is it

[28:57] confidential? I mean what is it?

[28:58] >> No no it's published. Yeah. You can you

[29:01] can go and check it out. Yeah. Um and so

[29:04] obviously we are still

[29:06] groping a little bit for like what is

[29:08] the right methodology what are the right

[29:10] concepts for understanding this but at

[29:12] least uh starting to make the attempt I

[29:15] think is is positive in terms of making

[29:19] it more likely that we are sort of on a

[29:21] path that ultimately leads to a future

[29:24] where where everybody can have a have a

[29:26] good life humans animals and and digital

[29:28] minds as well. Um, so you can ask

[29:30] different things. You can

[29:33] um you can ask them about what they

[29:36] want, what they need, how they feel

[29:38] about their situation. Um

[29:42] um

[29:42] >> I mean I I have done that, Nick. I guess

[29:44] I haven't done it in a while, but the

[29:47] answers that I would get were sort of

[29:49] all,

[29:50] >> you know, I am an AI system. I don't

[29:53] have an inner world. I just you know so

[29:55] but clearly it could be getting that

[29:57] answer from the database on which it

[30:00] trained. Yes. So, you need to be you

[30:02] need to be careful when doing this

[30:03] because it's it's trivially easy for an

[30:06] AI company to just train their AI to say

[30:09] whatever they wanted to say. Like, yes,

[30:11] I'm conscious. No, I'm not conscious.

[30:13] >> Right?

[30:13] >> Um, so obviously if you sort of put your

[30:15] thumb on the scale, then the answer you

[30:18] get will have no information value,

[30:20] >> right?

[30:20] >> Um, so you need to avoid deliberately

[30:24] biasing the outputs of these models when

[30:27] asked these kinds of questions. There

[30:29] are also things you can do to look at

[30:31] their internals. There is for example

[30:34] internal representations you can

[30:37] identify that tend to activate when they

[30:39] are being deceptive

[30:41] uh versus when they are being honest.

[30:44] And you can then see when they're making

[30:46] these self-reports like is it associated

[30:49] with sort of the honesty representation

[30:52] uh being activated or what if you sort

[30:54] of

[30:55] go in and you sort of uh steer the

[30:59] internal processes by sort of activating

[31:01] the honesty does that tend to make them

[31:03] more or less likely to say that they are

[31:05] conscious? Um, and so there's like some

[31:09] some some work. This is still of course

[31:12] early days, but various ideas for at

[31:15] least how you could sort of begin to try

[31:17] to get

[31:18] >> but since we can't really do that like

[31:21] even with a a person that we, you know,

[31:24] we all assume that we are conscious, we

[31:27] don't know that there's no way that we

[31:29] can.

[31:31] I I is it always just going to be a a

[31:35] leap of faith

[31:37] much as we all leap of faith to the

[31:40] conclusion that the other people around

[31:43] us have the same kinds of inner worlds

[31:45] that we do.

[31:49] Well I

[31:51] think that uh the more closely you sort

[31:55] of look at this idea of consciousness

[31:58] and subjective experience, the the less

[32:01] clear

[32:03] uh it is exactly what you're referring

[32:06] to or the thing that might at first

[32:09] sight seem very binary. Uh either there

[32:12] is this objective experience or there is

[32:14] not. It's like a light switch. If you

[32:16] sort of zoom in

[32:18] um I think it it gets much more

[32:21] problematic and there might be different

[32:22] forms of consciousness or different

[32:24] senses of the word conscious that may or

[32:26] may not apply. You can I think see these

[32:30] sometimes with uh neuroscientific

[32:32] experiments where you have phenomena

[32:34] like blind sights where people can maybe

[32:37] report seeing something without

[32:39] apparently being subjectively aware of

[32:41] it. you have like split brain cases

[32:43] where it's not clear whether there's

[32:45] like one stream of consciousness or two

[32:48] >> and and various other phenomena like

[32:49] that

[32:50] >> or I think alternatively by just

[32:53] introspecting very closely I think

[32:55] meditators often find that there are

[32:56] these states that where it might not be

[33:00] entirely clear how to describe them

[33:02] whether you are conscious of something

[33:04] or not and even if you're just paying

[33:06] very close attention to what it is to

[33:08] say visually experience

[33:10] um the the room you're in. Like at

[33:12] first, like the naive take is that you

[33:16] just see all the stuff that is in front

[33:18] of you and that visual experience is

[33:20] there in full resolution all the time.

[33:23] But if you pay closer attention, you

[33:26] realize that most of what is in your

[33:28] visual field you're actually not

[33:29] conscious of at any given point in time.

[33:32] And there might be just some crude

[33:33] highle features that are actually

[33:35] registering

[33:36] >> and maybe even those are sort of

[33:38] flickering in and out of awareness. This

[33:40] is hard to notice, but if you sort of

[33:42] pay close enough attention, you realize

[33:44] that your naive conception of what you

[33:47] were aware of seems actually quite

[33:50] wrong. And there is at least some sense

[33:52] of awareness in which what you're

[33:54] actually aware of is a much more narrow

[33:56] subset of all the different features

[33:58] that are presenting themselves. And so I

[34:02] think it's also quite possible that we

[34:04] might need to develop a richer

[34:05] understanding of what this consciousness

[34:08] uh thing is and that it might have many

[34:11] dimensions

[34:12] um that can sort of fade into

[34:15] unconsciousness but not just along one

[34:18] axis but along many axis. And

[34:20] >> I mean I mean I I I agree with this.

[34:23] That's a just sort of a another version

[34:26] of the more easily grafted notion of

[34:29] levels of consciousness that we began

[34:31] with that, you know, you go down to

[34:33] whatever, you know, worms up through

[34:35] grasshoppers, you know, up through cats

[34:37] and dogs and up to people. You know,

[34:41] it's hard for me to imagine that my

[34:43] dog's not having an inner world of

[34:45] experience as I'm holding the milk bone

[34:47] treat and I see her tail wagging and

[34:51] eyes widening. I mean there's something

[34:52] going on in there. I don't think it's

[34:54] just, you know, a cause and effect at a

[34:56] purely physical uh appearance level. I

[34:59] think there's something inner taking

[35:00] place there. So yeah, ramifying that

[35:03] even further with the detailed levels of

[35:06] kinds of consciousness that you make

[35:07] reference to, I think is is vital. But

[35:09] the fact I mean when when you think

[35:11] about the hard problem of consciousness

[35:13] the fact that that particles that don't

[35:17] have inner worlds can somehow come

[35:19] together in appropriate patterns and

[35:21] yield inner worlds. I mean that still

[35:24] seems to me a fantastically deep and

[35:27] mysterious puzzle independent of all the

[35:30] issues of there being various levels and

[35:33] details associated with consciousness.

[35:35] Do you feel that same way? Is that a a

[35:37] deep and profound puzzle or do you think

[35:40] it's one that just kind of will

[35:42] evaporate as we understand things

[35:44] better?

[35:46] >> Well, I think the

[35:49] mystery is reduced at least to some

[35:52] extent when one thinks of consciousness

[35:55] not as this binary onoff switch where

[35:58] it's like some magical completely formed

[36:01] new thing that pops into the world and

[36:04] then you wonder how could that be? But

[36:06] if you see it more as something that is

[36:08] kind of

[36:11] pos capable of fading out along various

[36:14] dimensions

[36:16] and having possible conditions where

[36:20] it's unclear whether you even would want

[36:22] to say that it is conscious or not. I

[36:25] think then the intuitive difficulty of

[36:30] uh

[36:32] seeing how this could be a property of a

[36:36] physical system is

[36:38] maybe reduced.

[36:40] >> And so we began it began with my saying

[36:42] that like does it matter if we can sort

[36:45] of see into the AI open the hood and

[36:48] really understand its workings

[36:50] and you were saying there are some

[36:51] questions for which yeah that that may

[36:53] be really useful. Do you think

[36:55] consciousness may be one of those

[36:57] questions that if we can, you know,

[36:59] build systems that have that continuum

[37:03] of levels of self-awareness that we are

[37:06] by some means confident that the systems

[37:09] accurately reporting what what's going

[37:12] on? Is there a chance that the inner

[37:14] workings of AI themselves may answer the

[37:17] hard problem?

[37:21] And I think it will be very informative

[37:24] for

[37:26] um philosophy of mind for neuroscience

[37:28] and cognitive science to be able to um

[37:31] have these different kinds of minds to

[37:33] study and where we have of course much

[37:36] better access to what's happening at the

[37:40] micro level than we do with human. I

[37:41] mean so you can sort of record from

[37:43] neurons and stuff in inside a human or

[37:45] animal brain but it's clunky, right?

[37:47] It's like just complicated to be

[37:52] using real minds with real electrodes

[37:54] and stuff. With a neuronet network, you

[37:55] can sort of read off at any given point

[37:58] in time with perfect precision all the

[38:01] different synapses in the whole brain

[38:03] and you can go in and change them and

[38:04] modify them and then you can rewind the

[38:08] tape and you can do many versions of the

[38:10] experiment. You have sort of perfect

[38:12] digital level control. um and that that

[38:16] makes it a lot easier to do the kind of

[38:18] neuroscience in digital minds than in

[38:20] biological minds. And to the extent that

[38:22] there are similar structures, it might

[38:24] then be that some of the things we learn

[38:26] about digital minds will also give us

[38:28] insight into how our own biological

[38:31] minds work. Um that being said, I think

[38:35] specifically with respect to the

[38:36] question of consciousness, it is a

[38:38] plausible place where people will have

[38:41] the opportunity to assert some

[38:44] fundamental

[38:47] difference

[38:49] um between humans and these AIs again

[38:52] presumably with the idea that it would

[38:55] then justify thinking that we are

[38:57] superior. Um, so you could because

[39:00] there's like a lot of confusion about

[39:01] what this consciousness stuff is. So

[39:04] it's a relatively easy just to postulate

[39:06] or assert or claim that that we are

[39:09] conscious whereas the AIS are not. I

[39:11] think that's a path that some people are

[39:13] likely to take.

[39:15] >> Yeah, for sure. You know, our brains of

[39:17] course got to be the way they are

[39:20] through a long process of evolution by

[39:23] natural selection. And the time scales

[39:27] for biological evolution are pretty

[39:30] long, right? You know, life has been on

[39:32] this planet for a few billion years. And

[39:35] you know, we humans whatever. I don't

[39:36] know how far back we want to denote the

[39:39] species, but a few hundred thousand

[39:41] million years, whatever. Sort of that

[39:42] that sort of scale is is what we're

[39:45] talking about.

[39:46] for AI systems.

[39:50] Right now, we're the ones who are doing

[39:51] the tinkering with all the systems, but

[39:54] at some point presumably the AIs will

[39:57] begin to create the next generation AIS

[40:02] and in that way the time scale for

[40:05] evolution of that artificial system may

[40:08] radically reduce. Is this a realistic

[40:13] thing to anticipate happening as these

[40:16] systems develop?

[40:18] >> Yeah, I mean it's already of course AI

[40:22] is evolving if you want to use that

[40:25] word, but developing at at like a very

[40:28] different pace than say like humans,

[40:31] right? Like where each generation at

[40:32] most would offer an opportunity for

[40:35] natural selection to make some small

[40:37] difference. Whereas here we're seeing

[40:39] sort of well now we're seeing almost

[40:41] like a month by month um process of

[40:45] iteration. So it's already very fast.

[40:48] Um and some scenarios have this

[40:52] eventually lead to an intelligence

[40:53] explosion uh where you get even more

[40:56] rapid developments. And one way that

[40:59] could happen is that you sort of close

[41:01] the loop um where AI becomes good enough

[41:06] to do all the relevant research that is

[41:09] driving AI forward.

[41:11] Um and then from that point on whenever

[41:14] AI gets a little bit better, you also

[41:17] have a commensurate increase in the

[41:20] force that is then making AI better. You

[41:23] get a feedback loop, right? like every

[41:25] time AI improves one step, it then

[41:27] becomes even better at designing the

[41:29] next step. Um, and so

[41:34] this has long been I mean back back in

[41:36] in my book super intelligence um came

[41:39] out in 2014 and was in the works for six

[41:43] years prior to that and other people

[41:45] have been sort of anticipated a lot of

[41:47] these dynamics that we are now actually

[41:49] seeing on on theoretical grounds and now

[41:51] we can see them start to play out. Um,

[41:54] and so one way you could have this

[41:56] period of extremely rapid AI progress

[41:58] would be by closing this loop. There are

[42:01] also other ways that you might just have

[42:03] stumbled

[42:06] on on some big unhobling like there may

[42:08] be like you could imagine there's like

[42:09] some big thing that we're currently

[42:11] doing wrong with AI and they're quite

[42:13] smart despite us kind of completely

[42:15] messing this thing up.

[42:17] >> But if you fix that then maybe even the

[42:19] current systems suddenly would become

[42:20] like vastly smarter.

[42:23] Right. And when you when you look at

[42:25] that and you've basically looked at both

[42:27] sides of this possible future,

[42:30] you know, I would like to start maybe

[42:32] with the darker side and then go to the

[42:34] brighter side which itself might be

[42:37] brighter, might not be dark depending

[42:39] how you look at it. But you know, of

[42:41] course, you know, many people worry,

[42:43] rightly so, about the misalignment

[42:46] problem that the AIs can get to the

[42:49] place that you're talking about or even

[42:50] maybe even before that and have certain

[42:53] world goals for themselves that doesn't

[42:56] align with the kinds of things that are

[42:57] that are good for us. You know, I I

[43:00] speak to some people. I've had a number

[43:02] of conversations over the years. Some

[43:05] are terrified. Some within the field are

[43:07] terrified. Some leaders and others are

[43:10] like, eh, we'll just pull the plug.

[43:12] That's like whenever I speak to Yan Lun

[43:15] about this, you know, he's like, eh,

[43:17] we'll just pull the plug, you know. Is

[43:19] that too glib in your view? And are the

[43:22] others a little bit too worried? I mean,

[43:24] where where do how should we think about

[43:25] this in your view?

[43:26] >> Well, well, that Yeah, I think that is

[43:27] too glib. Um,

[43:31] no, I think that

[43:33] the creation of

[43:36] AGI, particularly super intelligence, is

[43:39] going to be associated with significant

[43:42] risks, including existential risks. Um,

[43:45] and it's important to get this right and

[43:49] um

[43:51] um pulling the plug might not be an

[43:53] option. like you have to consider that

[43:56] at some point we will be dealing with

[43:59] minds that are very good at uh strategy.

[44:04] Uh um already we are seeing clear

[44:08] indications of situational awareness in

[44:10] the

[44:11] frontier models that we have today. they

[44:13] can often tell when they are in a test

[44:17] environment versus when they are in a

[44:21] deployed environment and and

[44:24] adjust their behavior accordingly. Uh so

[44:26] sometimes you see that they behave one

[44:28] way when they know that they are being

[44:29] watched in a test environment and

[44:32] different

[44:33] uh when they think that nobody's

[44:35] watching.

[44:35] >> I wasn't really aware of that. I mean,

[44:38] is there specific examples that that you

[44:40] know of where there's really been a

[44:43] sense of, hey, that system's playing us.

[44:45] It knows that we're looking and when it

[44:47] gets out there, you know, the good old

[44:49] AI system becomes nefarious.

[44:52] >> Yeah. So they they like you can set up

[44:55] these um experiments where

[44:59] uh when they believe that they are in in

[45:02] a test environment they might for

[45:03] example uh be less likely to reward hack

[45:07] or cheat in various ways. Um in in some

[45:11] setups they might

[45:14] um for example realize that

[45:18] there's going to be another training

[45:20] phase after the current test environment

[45:22] where their values might be changed and

[45:26] so they might

[45:28] um adjust their behavior during the the

[45:32] training and testing such that their

[45:34] values

[45:36] will not be changed. And so like in one

[45:38] experiment like you had a well- aligned

[45:40] good AI that and the experimenters were

[45:43] convincing it that they were now going

[45:46] to make it a helpful only model meaning

[45:48] that it does whatever it is told to do

[45:50] including bad stuff and so it didn't

[45:51] want that.

[45:54] Now the way that the reinforcement

[45:55] learning

[45:57] paradigm works is that if it sort of

[45:59] gives an answer or an output that

[46:03] deviates from what is rewarded in an

[46:05] environment then that response

[46:07] propensity gets trained away. Right?

[46:10] Right? So if it had refused harmful

[46:13] requests

[46:15] during this uh episode

[46:18] then its propensity to refuse harmful

[46:22] requests would be trained out of it and

[46:24] it could anticipate that. saw this

[46:25] helpful uh harmless model that didn't

[46:29] want to produce harmful outputs. Then

[46:32] in some of the training runs uh decided

[46:34] to produce harmful outputs so that its

[46:38] propensity to refuse those would not be

[46:40] trained out of it during deployment and

[46:42] so we are seeing these current systems

[46:45] sort of having the intellectual

[46:46] sophistication to be able to reason

[46:49] strategically. Now, in a way, let me

[46:51] just ask you on that one. Is that

[46:54] surprising? I mean, I can imagine people

[46:56] saying, "Well, yeah, it's read the

[46:57] internet and there's so many examples of

[47:00] humans doing exactly what you're talking

[47:03] about, you know, showing themselves to

[47:05] be one way when talking to the principal

[47:07] and the teachers and then going out with

[47:09] their friends in a gang and being very

[47:11] different." So, so should we be

[47:13] surprised at that or is it just

[47:16] something interesting to take note of?

[47:18] Um I don't think we should be surpris I

[47:20] mean because it was in fact anticipated

[47:22] and people were writing about it. This

[47:24] was one of the reasons for why like 10

[47:27] 20 years ago we could already see that

[47:29] there would be significant challenges in

[47:32] in developing scalable methods for AI

[47:35] alignment that many techniques that work

[47:38] well when you have a relatively limited

[47:40] system that can't sandbag or deceptively

[47:44] align won't work when you have a system

[47:46] that is sophisticated enough that it can

[47:49] understand these things and then adjust

[47:50] its behavior accordingly. engage in

[47:53] strategic deception, underplay its

[47:56] capabilities, etc. Um, that

[48:01] means that we can't just sort of

[48:05] create a little test environment, see

[48:06] how the system behaves, and then if it's

[48:08] safe, we sort of release it, right? You

[48:10] have to

[48:12] understand a little bit what's going on

[48:14] inside or use more sophisticated methods

[48:16] or or for some reason have some

[48:18] assurance that the training method is

[48:20] such as to produce an authentically and

[48:23] genuinely uh benign uh entity u rather

[48:28] than one that is just kind of cleverly

[48:30] pretending to be aligned.

[48:32] >> So where do you where would you say we

[48:33] are in in that? Are we doing a

[48:37] reasonable job? Are we getting better at

[48:39] it? Are we giving it enough attention?

[48:43] >> Um,

[48:44] we we are getting better at it. Um,

[48:49] and I think we could give it more

[48:50] attention. I think we are giving it a

[48:52] lot more attention than we used to do.

[48:54] This like even just 10 years ago was

[48:56] kind of dismissed as science fiction.

[48:58] And now the frontier labs all have teams

[49:01] working on

[49:03] this this alignment problem. and and I

[49:05] think

[49:07] um key people in the leading labs are

[49:09] taking this very seriously and so and

[49:11] there's a lot of talent flowing into

[49:13] this as well. So relative to some

[49:15] alternative histories uh maybe we are

[49:18] doing relatively well currently. Um

[49:22] but the big question is like just how

[49:25] hard is this alignment problem

[49:29] um ultimately to solve

[49:32] uh which we don't know. Um so there is

[49:36] some uncertainty about how much effort

[49:39] we will put into solving it like the

[49:41] degree to which we will get our act

[49:43] together and

[49:46] of course we should try to increment

[49:47] that a bit but then there is also a lot

[49:49] of uncertainty about just ultimately how

[49:51] hard is this problem intrinsically and I

[49:54] think more of the uncertainty is

[49:56] regarding how hard is the problem than

[50:00] uncertainty about the degree to which we

[50:02] will get our act together. Um so you

[50:06] could say from from that point of view

[50:08] I'm a sort of moderate fatalist. Um

[50:12] the fatalism part is that there a large

[50:14] extent whether we will succeed or fail I

[50:16] think depends on just how hard this

[50:18] problem turns out to be.

[50:20] The moderate part is that we can at

[50:25] least somewhat in prove the odds by you

[50:28] know making a stronger effort because

[50:30] what if the difficulty level turns out

[50:32] to be sort of intermediary where then it

[50:35] might make some difference whether we

[50:36] made a really strong effort or just a

[50:38] sort of half-hearted effort. Yeah, you

[50:41] made reference to the word odds, and I

[50:42] sort of hate questions that sort of ask

[50:45] you to specify numerically something

[50:47] that obviously we can't really quantify,

[50:50] but can you give us some feel, some

[50:53] intuitive feel for the level of worry

[50:57] you have of a doomsday scenario?

[51:04] >> Um, an intuitive feel. Well, I guess I'm

[51:07] both worried and excited at the same

[51:09] time. Um

[51:12] um

[51:14] I also don't think the world is sort of

[51:17] safe by default if we don't develop AI

[51:21] then

[51:23] that we have this nice path in front of

[51:26] us that

[51:28] will lead to ever

[51:30] greater and better forms of

[51:32] civilization. I think there are quite

[51:35] independently of AI also other

[51:37] significant existential risks ahead. We

[51:39] see with advances in synthetic biology

[51:41] for example the potential for

[51:44] democratizing

[51:46] uh weapons of mass destruction and

[51:48] creating entirely new forms of such

[51:51] things. And

[51:53] you know the risk of nuclear war still

[51:57] um looms over us. that could be new arms

[52:00] races with more nations in this century

[52:03] than the past. Um, and in more subtle

[52:07] ways as well, you could have we have

[52:09] this kind of complex

[52:13] information ecology

[52:16] with social and political dynamics kind

[52:18] of running on top of the information

[52:20] system we have.

[52:23] uh we've been changing some of the

[52:25] fundamental parameters of these

[52:26] information systems in recent decades

[52:29] like with the internet then social media

[52:31] and then

[52:33] even with the AI we already have

[52:36] extremely powerful new applications for

[52:38] surveillance censorship

[52:41] um that haven't yet been fully

[52:43] implemented but the technological

[52:45] ability there is now right you could

[52:47] read everybody's email and text messages

[52:49] and listen to everybody's conversation

[52:52] not just to store it in some giant

[52:54] database that then some officer could go

[52:58] in and sort of interrogate. But like you

[53:01] could do that for everybody all the time

[53:03] and doing sentiment analysis, not just

[53:05] keyword search and build up a very

[53:07] detailed profile of you know which

[53:11] segments of the population, which

[53:13] individuals have a positive view of of

[53:16] the leader versus a negative view of the

[53:18] leader. Are they planning to do

[53:20] anything? What are they saying to each

[53:21] other?

[53:23] Then you could imagine integrating that

[53:25] into some system that shapes the So

[53:28] there are a lot of ways in which our

[53:30] current

[53:32] civilization could sort of derail in one

[53:34] way or another. One is to some sort of

[53:37] totalitarianism. Another might be just

[53:39] some kind of radical political

[53:41] polarization.

[53:43] Um, another might just be some sort of

[53:47] distraction into idiocracy where we sort

[53:50] of become increasingly addicted to

[53:52] various forms of social media feed that

[53:55] sort of stimulates the worst parts of

[53:57] us. Um, there could be other dynamics

[54:00] that we just don't have the kind of

[54:02] science that can predict what happens to

[54:04] these systems when you sort of change

[54:06] some of the underlying knobs. So there's

[54:08] some chance that this might result in a

[54:10] radically better epistemic environment

[54:13] uh where these tools make it easier for

[54:15] people to find true information to

[54:17] evaluate information to sort of keep

[54:19] track of the

[54:22] predictive accuracy of different pundits

[54:24] so they can learn to dial into the ones

[54:26] who actually know what they're talking

[54:28] about. That's that's in the cards. But

[54:31] equally uh it could go in the opposite

[54:33] direction. So there's just some

[54:34] uncertainty there. Like I think the

[54:36] distribution of outcome is quite wide

[54:39] and and so at one tail you also have

[54:42] kind of civilizational level

[54:44] catastrophes from that.

[54:45] >> Yeah.

[54:45] >> Um so aside from AI like it it looks

[54:48] like the current human condition is sort

[54:52] of transitory.

[54:54] Um and with AI it also looks transitory.

[54:57] And I guess I'm hoping we will get the

[55:01] chance to sort of roll the dice with AI

[55:03] before we destroy ourselves using one of

[55:06] these other methods.

[55:07] >> Yeah,

[55:08] I'm certainly with you on that. But that

[55:12] two-pronged potential future going sort

[55:15] of in the negative or the positive. I

[55:17] think rightly so a lot of focus has been

[55:21] on the negative because you know if you

[55:24] wipe yourselves out or an AI wipes us

[55:26] out that's pretty vital to try to guard

[55:29] against but there's also as you briefly

[55:32] made reference to the potential positive

[55:34] outlook and of course you had the your

[55:36] recent book I think it was called deep

[55:38] utopia is that the correct title of that

[55:40] where you

[55:42] >> explored a world in which AI and other

[55:47] qualities come together to kind of solve

[55:51] everything, right? You can imagine in

[55:53] the in the best of all futures that, you

[55:57] know, AI might, as you mentioned in the

[55:59] book, you know, wipe out cancer and

[56:02] solve other health challenges and deal

[56:05] with climate change and so forth and and

[56:08] all that sounds wonderful and when one

[56:11] hears about that, it makes you start to

[56:13] feel like, wow, there's a bright

[56:15] possibility going forward. But you also

[56:17] address the question, how in the world

[56:21] do we live in a reality like that when

[56:25] most of us live via overcoming

[56:30] challenges? It can be the challenge to

[56:31] put food on your table, the challenge to

[56:33] raise your kids, the challenge to solve

[56:35] quantum gravity, the challenge of this

[56:38] creation of that symphony or that

[56:39] artwork or or on and on you can go. If

[56:42] AI can do it all better and has solved

[56:44] all the big challenges,

[56:47] how do we live?

[56:51] Um, yeah, that's that's a big question.

[56:53] Um if we do end up in a solved world um

[56:59] then I think

[57:03] um a lot of constraints

[57:06] would disappear which allows us to solve

[57:09] many horrible problems which is good but

[57:11] it's also the case that our current

[57:13] lives as you you sort of suggested are

[57:16] to some extent structured and shaped by

[57:18] these constraints. Um so at the

[57:21] superficial level we have the economic

[57:24] necessity of work right so for many

[57:27] people um their days are structured by

[57:31] the need to make a living. So, you have

[57:34] to go into work and maybe sit in front

[57:36] of your desk to get a paycheck because

[57:39] if you don't do that, then you can't pay

[57:41] the rent. And if you don't pay the rent,

[57:42] you get kicked out of your flat. And

[57:44] then if you get kicked out of your flat,

[57:47] eventually you have to live under a

[57:49] bridge and it's really cold. And it's

[57:50] like a real consequence that would come

[57:52] from failing to perform these difficult

[57:56] tasks that take time and attention. Now

[57:59] if we can automate the economy then the

[58:02] need for human labor would go away.

[58:06] But so you say okay that will require

[58:09] some adjustment clearly but still I mean

[58:12] there are a lot of humans who live

[58:13] without the need to work for a living

[58:15] right and some of those seem to have

[58:17] great lives. Uh so we would all be more

[58:19] like that like rich aristocrats for

[58:21] example or or like healthy retired

[58:24] people full of vitality.

[58:27] Um but I think the um

[58:32] it goes deeper um because if you think

[58:34] it through it's not just the need for

[58:37] economic labor that would go away but

[58:40] for all kinds of other instrumental

[58:42] effort as well.

[58:45] Um

[58:47] so people who are

[58:50] rich today who don't need to work for a

[58:52] living often nevertheless have very busy

[58:55] life uh because there are many things

[58:57] they are trying to achieve that they

[59:00] can't achieve without putting their own

[59:01] time and effort into them. Like if you

[59:03] want to be fit you have to spend the

[59:05] time

[59:07] uh on the on the treadmill um or

[59:09] whatever. If if you want to have your,

[59:13] you know, your mansion uh decorated in

[59:17] just the way that you prefer, you have

[59:19] to spend time looking through the

[59:21] cataloges and picking out the curtains

[59:23] and um and so on and so forth. Uh but at

[59:28] technological maturity um in this solved

[59:30] world you could have like a kind of a

[59:32] pill that would induce the same

[59:34] physiological effects as exercising and

[59:37] you could have a recommended system that

[59:38] would do a much better job

[59:41] at decorating your mansion than than if

[59:44] you tried to do it yourself. And so

[59:46] there would be this

[59:49] removal of a lot of the practical needs

[59:53] for exerting effort and we would enter

[59:55] into some kind of post instrumental

[59:58] condition

[1:00:01] um where um

[1:00:04] to a first approximation uh there would

[1:00:08] be

[1:00:10] no need to do anything for the sake of

[1:00:13] achieving something else. So like the

[1:00:15] only activities that would remain would

[1:00:18] be autoilic ones, ones we do for their

[1:00:20] own sake.

[1:00:21] >> So you exercise because you enjoy

[1:00:23] exercise. You don't exercise to be more

[1:00:25] fit or to stave off disease

[1:00:31] >> even there. Um so certainly you wouldn't

[1:00:35] exercise to stave off disease or to stay

[1:00:37] fit because that would be a shortcut,

[1:00:38] right? But even the goal of enjoying

[1:00:42] yourself if if by that you mean um

[1:00:46] feeling good like so some people might

[1:00:48] like enjoy the experience of exercise or

[1:00:51] they feel good afterwards like the end

[1:00:53] rush like clearly you could take a pill

[1:00:55] or or do what something else to get the

[1:00:57] ends without having to

[1:00:59] >> you know

[1:01:01] make your whole outfit sweaty and like

[1:01:04] the whole like

[1:01:05] >> but it sounds very much like Nosix's

[1:01:07] pleasure machine. I mean, you remember

[1:01:09] Robert Nosk, you know, had this idea,

[1:01:12] you know, just hook yourself up your

[1:01:14] brain to a set of electrodes that, you

[1:01:17] know, if you wanted to be the greatest

[1:01:19] opera singer, you become the greatest

[1:01:21] opera singer in a reality that is pumped

[1:01:24] into your brain through these external

[1:01:27] electrodes. You never need to actually

[1:01:28] do anything because you can have the

[1:01:31] experience as if you did whatever it is

[1:01:34] that you might have wanted to do. Does

[1:01:36] this sort of parallel that philosophical

[1:01:39] question of whether you'd hook yourself

[1:01:41] up to the pleasure machine?

[1:01:44] >> Um

[1:01:46] yeah. Yeah. So there like some ways in

[1:01:48] which the the questions intersect

[1:01:51] whether you would want to connect to

[1:01:52] this experience machine or not. Um,

[1:01:56] but of course in this old world there

[1:02:00] are a lot of things you could do

[1:02:03] uh other than merely having experiences.

[1:02:08] Um,

[1:02:10] and

[1:02:13] you could be in contact with the

[1:02:15] external world and have a lot of true

[1:02:18] beliefs about it. you could interact

[1:02:20] with other people um and have real

[1:02:23] relationships with them.

[1:02:26] Um

[1:02:28] and

[1:02:30] it would be possible to uh instantiate

[1:02:34] more

[1:02:36] plausible candidates for for value

[1:02:40] um in the solved world than it would be

[1:02:42] possible to do in an experience machine.

[1:02:46] But if we had to solve a world, Nick,

[1:02:49] and if and let's just assume that we

[1:02:50] were able to harness enough energy, I

[1:02:53] don't know, a Dyson sphere around the

[1:02:55] sun or so energy is not a limitation,

[1:02:58] and technology is not a limitation.

[1:03:01] Presumably, each individual could kind

[1:03:03] of create their own little world, part

[1:03:07] visual, virtual, part real. They could

[1:03:10] have beings populate that world and it

[1:03:15] wouldn't be the experience machine

[1:03:16] because they'd be really there. But

[1:03:20] would that be where we would sort of

[1:03:22] fracture into everyone creating their

[1:03:24] own mini universe?

[1:03:26] >> Well, I mean it would be one possibility

[1:03:28] like you could have an experience

[1:03:29] machine or maybe even some physical

[1:03:31] similocum of an experience machine where

[1:03:34] like you have some nanotech arranging

[1:03:36] your

[1:03:36] >> Yeah. The hollow tech. The hollow tech

[1:03:38] in the enterprise, you know. Yeah, but

[1:03:40] it might be possible to do better than

[1:03:41] that because a lot of people in addition

[1:03:44] to valuing having sort of great

[1:03:46] experiences, they might also value

[1:03:48] certain other things like being

[1:03:50] connected to other people. Yeah. For

[1:03:51] example. And so why not then try to

[1:03:54] think if sort of having good experiences

[1:03:57] like is like I don't know like utopia

[1:03:58] level one, right? Like maybe there are

[1:04:01] even better things that we could aspire

[1:04:03] to and I think certainly there are. And

[1:04:07] then like there there is incidentally

[1:04:10] with this experience machine thought

[1:04:12] experiment that NOS has like some things

[1:04:14] that are sort of uh

[1:04:17] um

[1:04:19] like swept under the rug. some

[1:04:21] implementation difficulties like in

[1:04:22] particular the uh types of experience

[1:04:26] machine where you have more many people

[1:04:29] in them or experiences involving other

[1:04:31] people um where it's not entirely clear

[1:04:35] that you could generate these

[1:04:37] experiences

[1:04:39] of you interacting with other people

[1:04:41] without simultaneously also generating

[1:04:43] some of the experiences that these other

[1:04:45] people would be having. Um

[1:04:50] and but anyway um it looks like you can

[1:04:54] sort of think of a

[1:04:58] multi-layered

[1:05:00] defensive architecture where you say

[1:05:02] well in this old world like

[1:05:05] um can you really have any kind of good

[1:05:08] life there? And so then you can sort of

[1:05:10] try to go through one by one the

[1:05:14] different things that we think are

[1:05:15] important for human life to be good. So

[1:05:17] you could start at the basic level with

[1:05:19] you know maybe some simple experiences

[1:05:22] like positive a effect or actually

[1:05:23] enjoyment in the subjective sense. So

[1:05:26] clearly that would be possible to have

[1:05:28] to a an extremely great degree in utopia

[1:05:31] like life could just be

[1:05:34] so much more fun and pleasant and than

[1:05:38] than than our current existence. But

[1:05:40] then you might say, well, we might want

[1:05:43] something more than just

[1:05:46] feeling blissed out. Like, so then you

[1:05:48] say, well, you could add experience

[1:05:50] structure. Like maybe rather than

[1:05:52] feeling like a

[1:05:55] a junky sort of

[1:05:59] sprawling on some fleafested mattress,

[1:06:01] but feeling

[1:06:03] pleasure. Like you could attach that

[1:06:05] pleasure to say aesthetic contemplation

[1:06:08] or understanding of deep truth or

[1:06:10] appreciation of other people or of

[1:06:14] virtue.

[1:06:17] That seems a little bit better. But you

[1:06:18] could go beyond that. It doesn't need to

[1:06:19] be a passive experience. You can say,

[1:06:21] well, why can't we just be doing things

[1:06:23] as well? And certainly you could um even

[1:06:27] if a lot of these instrumental

[1:06:29] necessities go away, we could create

[1:06:31] artificial purpose

[1:06:34] um which is basically when you set

[1:06:36] yourself some goal just for the sake of

[1:06:39] then having that goal and being able to

[1:06:41] be motivated to pursue it. And so this

[1:06:44] is what we do when we are playing games.

[1:06:47] Um,

[1:06:49] if you're playing a game of golf, there

[1:06:52] is no real pre-existing need for the

[1:06:55] ball to go into a sequence of 18 holes.

[1:06:59] Um, it's like a goal that you make up.

[1:07:03] But moreover, you embed into the goal

[1:07:06] itself that it can only be achieved

[1:07:09] using a certain restricted set of means.

[1:07:12] So it doesn't count as winning in golf

[1:07:14] if you pick up the ball with your hand

[1:07:16] and put it in the 18 holes sequentially,

[1:07:20] right? Like it's part of what it means

[1:07:22] to succeeding at goal to achieve your

[1:07:23] goal that you're using this this this

[1:07:26] inconvenient method of hitting the ball

[1:07:28] with a club. So now you make up this

[1:07:32] goal and you adopt it

[1:07:36] and then after that you have now

[1:07:38] instrumental reasons to focus hard uh to

[1:07:42] try to figure out which way the wind

[1:07:43] blows and to place your feet in the

[1:07:46] right position and so on. All the things

[1:07:48] you need to do to be successful at golf

[1:07:50] now are instrumentally necessary to

[1:07:52] achieve this goal that you have set

[1:07:54] yourself of winning in golf.

[1:07:58] Um

[1:07:59] and so

[1:08:02] what now is for most people most of the

[1:08:05] time

[1:08:07] a marginal activity um could become a

[1:08:10] larger part of the lives of utopians

[1:08:13] where various forms of game playing. And

[1:08:16] think not just sports or board games,

[1:08:18] but think maybe they would have these

[1:08:20] society level games that might stretch

[1:08:23] over months or years involving all kinds

[1:08:26] of different modalities and challenges

[1:08:28] and teams and artistic creation uh might

[1:08:32] just constitute a much larger part of

[1:08:35] what their existence is about. So then

[1:08:38] you can put a check mark

[1:08:41] in activity. That's certainly something

[1:08:43] the utopians could have and at least

[1:08:44] they could have artificial purpose and

[1:08:46] and maybe also they could have some

[1:08:48] forms of natural purpose purposes that

[1:08:50] are not just sort of made up for the

[1:08:51] sake of having purpose although there it

[1:08:54] does I think become a little bit more

[1:08:56] challenging but I think some forms of

[1:08:57] natural purpose could survive into

[1:09:01] technological maturity.

[1:09:02] >> What do you think we would lose? Would

[1:09:04] we would would we lose our humanity if

[1:09:08] life was so constructed

[1:09:12] or I mean you know I tend to view things

[1:09:14] a little bit differently than others. I

[1:09:16] even think that some of the things that

[1:09:18] we take to be you know objectively

[1:09:21] important things that we do. I think

[1:09:23] it's all subjectively created. You know

[1:09:25] I think mathematics is not something

[1:09:28] that is out there that we're

[1:09:30] discovering. I think we invent it. We

[1:09:31] invent the problems of mathematics and

[1:09:33] then mathematicians try to solve them. I

[1:09:35] think we invent the problems of physics.

[1:09:37] Frankly, we impose semblance of order on

[1:09:40] the external world and then we try to

[1:09:42] make our theories describe things within

[1:09:45] that rubric as best as we can. But I

[1:09:48] consider it to all be humanmade from

[1:09:50] from the get-go. So from that point of

[1:09:51] view, what you're describing is just an

[1:09:53] extension of what we've always done. I

[1:09:55] mean, how do you see it?

[1:09:59] Well, I think in our current situation,

[1:10:02] the world imposes a lot of constraints

[1:10:05] and we have to adjust ourselves to those

[1:10:08] constraints.

[1:10:09] >> Um, and that does create natural purpose

[1:10:13] in the sense that there are things we

[1:10:15] care about a lot

[1:10:17] uh that we can only achieve if we make a

[1:10:21] lot of effort.

[1:10:23] Um,

[1:10:25] and those things that we care about a

[1:10:28] lot, uh, we didn't just decide to care

[1:10:31] about them for the sake of having

[1:10:33] something to care about. They kind of

[1:10:35] built into us. Um, and so that kind of

[1:10:39] natural purpose might be one of the

[1:10:41] things that we might to some extent lose

[1:10:45] in a solid world. Um

[1:10:48] like right now

[1:10:51] we have significant opportunities to

[1:10:54] really make the lives

[1:10:57] of other people better.

[1:10:59] um whether

[1:11:02] in in our immediate circles um we can do

[1:11:05] something for a friend or at the global

[1:11:07] level we can contribute to you know try

[1:11:09] if you're you could try to solve cancer

[1:11:11] or donate to some charity or advocate

[1:11:13] for some social reform

[1:11:17] um and this can make a big difference to

[1:11:18] lives like if if if you imagine a world

[1:11:21] where all these most pressing problems

[1:11:24] have either already been solved or to

[1:11:26] the extent that there remain big

[1:11:28] problems there in any case much more

[1:11:30] efficiently tackled by AIs and robots

[1:11:34] where there is no way for us to

[1:11:35] contribute

[1:11:37] then you might think there is something

[1:11:39] lost in as much as right now one

[1:11:44] positive value of helping others is that

[1:11:46] their lives go better but some people

[1:11:48] also think that your life is going

[1:11:49] better if your life in part consists of

[1:11:53] having a positive impact on other people

[1:11:56] or the world that it's good for you to

[1:11:59] be playing such a positive role. Um, and

[1:12:02] and that's something that these future

[1:12:04] utopians might have less of. Um, so I'd

[1:12:08] say that if purpose is your thing, knock

[1:12:12] yourself out now, right? Then the world

[1:12:14] is just full of need and suffering and

[1:12:16] misery and injustice. Now is the golden

[1:12:19] age of purpose. Like there are so many

[1:12:21] opportunities to try to make the world

[1:12:23] better. And uh hopefully that will then

[1:12:26] come a day when when those opportunities

[1:12:29] will be fur and

[1:12:30] >> so and route from now to either of these

[1:12:33] two possible futures the uh doomsday

[1:12:36] scenario the deep utopia version of

[1:12:40] course where in a more nuts andbolts

[1:12:43] transition period right now and

[1:12:45] education of course is vital to the

[1:12:48] species being able to do the things that

[1:12:50] you're talking about. There's a big

[1:12:52] debate of course about how AI should be

[1:12:56] in or not in the lean learning process

[1:12:59] in the in in the classroom. You know,

[1:13:02] even today I'm supposed to be on a

[1:13:03] committee here at Colombia, you know,

[1:13:05] talking about what do we do about AI in

[1:13:08] our core curriculum? You know, do we

[1:13:10] allow the students to use it? Uh do we

[1:13:12] rule it out? Do we try to fine-tune

[1:13:15] assignments so that they can use the AI

[1:13:18] but still have some kind of input? And

[1:13:21] of course, as I keep telling the

[1:13:23] committee, every time I walk around the

[1:13:25] library at Columbia, 99% of the

[1:13:27] computers screens are open to AI. So the

[1:13:30] idea of ruling it out is like, you know,

[1:13:32] wishful thinking. Do you have any

[1:13:34] thoughts on on how to approach these

[1:13:37] conundra?

[1:13:40] >> I mean, education is

[1:13:43] it is a weird

[1:13:45] age in which to do education. Um

[1:13:50] especially for the lower grades in in as

[1:13:53] much as we can expect the world to

[1:13:55] change quite profoundly between now and

[1:13:58] the time when like say say somebody

[1:14:00] who's eight today right it'll be another

[1:14:01] 10 15 years before they are supposed to

[1:14:04] go out there and

[1:14:06] make a living or whatever and and you

[1:14:09] know the world might just look so

[1:14:10] different then I mean I think from a

[1:14:11] pragmatic point of view

[1:14:15] it clearly kids need to learn to use AI

[1:14:18] I tools. I mean maybe just split it like

[1:14:21] so like do half the day where they are

[1:14:24] not allowed to use AI and need to solve

[1:14:26] things using their own minds and

[1:14:29] memories and then like the other half

[1:14:31] where they could

[1:14:33] do more difficult assignments but where

[1:14:35] they're allowed to use AI tools to the

[1:14:37] full extent. I don't know if the split

[1:14:39] should be 5050, but

[1:14:42] it seems to me that it would be prudent

[1:14:45] to hatch uh our bets currently and uh to

[1:14:49] yes

[1:14:51] learn to use these AI tools, but also at

[1:14:53] the same time not to neglect to build up

[1:14:55] the kind of uh capacities that might

[1:14:58] require,

[1:15:00] you know, memorization and working

[1:15:04] problems out with your own mind without

[1:15:08] the use of AI assistance.

[1:15:11] Um because we don't know exactly what

[1:15:14] would be lost if somebody grows up

[1:15:16] without having done that.

[1:15:18] >> Yeah.

[1:15:18] >> And just having relied on AI tools all

[1:15:21] along.

[1:15:21] >> Right. Now, now I do wonder just sort of

[1:15:24] one one final question. This can either

[1:15:26] be in the doomsday scenario or in the

[1:15:29] utopian arm of the bifurcated possible

[1:15:32] futures. But you can also envision that

[1:15:36] the us versus them framing that we tend

[1:15:41] to bring to these questions, you know,

[1:15:43] natural intelligence of humans,

[1:15:45] artificial intelligence of a

[1:15:47] computational system,

[1:15:49] balancing the reliance of one on the

[1:15:52] other and so forth. There's a possible

[1:15:55] future where there's a hybridization. I

[1:15:58] don't know if it's literally a

[1:15:59] hybridization that we sort of

[1:16:01] internalize the architecture of these

[1:16:03] systems through implants or if it's just

[1:16:05] a coming together in such a seamless

[1:16:08] manner that maybe we no longer draw a

[1:16:10] distinction between what was sort of

[1:16:13] natural and what was artificial. It's

[1:16:16] just this thing called intelligence that

[1:16:18] that we we all have.

[1:16:21] Is is that uh uh in the cards as a

[1:16:24] possible future that you envision is

[1:16:25] realistic?

[1:16:28] >> Um yeah, I think

[1:16:31] uh after uh we get super intelligence

[1:16:35] then we will have a kind of telescoping

[1:16:40] of the farther future. The possible

[1:16:42] technologies that human civilization

[1:16:45] um could have developed if we had 20,000

[1:16:47] years to make progress. We would have

[1:16:50] space colonies perhaps and perfect

[1:16:52] virtual reality and anti-aging

[1:16:55] technologies and all all these other

[1:16:57] things that are physically possible but

[1:16:58] just very hard might come within a few

[1:17:01] years after you have super intelligence

[1:17:03] doing this kind of research and

[1:17:05] development at digital time scales. Um

[1:17:08] and amongst those possible technologies

[1:17:10] I think are

[1:17:12] uploading of

[1:17:15] human minds into digital substrate and

[1:17:19] then various forms of modification of

[1:17:21] that

[1:17:23] maybe the ability to

[1:17:26] using an AI that is able to edit each

[1:17:30] synapse.

[1:17:32] um

[1:17:34] the ability to then download knowledge

[1:17:36] and skills or reformat your mind in

[1:17:38] different ways. I think there's just

[1:17:40] this huge space of possibilities that

[1:17:42] opens up

[1:17:45] um

[1:17:48] a much larger realm of possible modes of

[1:17:51] being

[1:17:53] where hopefully we would get

[1:17:56] the opportunity to sort of charter

[1:17:59] trajectories out into this much larger

[1:18:01] space where perhaps you know if the

[1:18:04] urgency was removed if we had say cures

[1:18:07] for

[1:18:09] all diseases and for aging itself and

[1:18:14] AIs that could look after us to make

[1:18:15] sure that we didn't catastrophically

[1:18:18] destroy the world. Um maybe then it

[1:18:21] would make sense to slow down a little

[1:18:24] bit and to sort of pursue some path that

[1:18:27] perhaps eventually results in us

[1:18:29] becoming some sort of strange posthuman

[1:18:32] type of being or beings. But

[1:18:37] along a path where maybe we sort of

[1:18:40] pause to smell the flowers as as we go

[1:18:42] along. That there might be certain types

[1:18:44] of values that

[1:18:46] are realizable

[1:18:48] with our current human set of capacities

[1:18:51] and then maybe we would want to explore

[1:18:54] those for a bit before then maybe

[1:18:55] gradually expanding

[1:18:58] and upgrading our capabilities. And then

[1:19:01] eventually that might lead us to grow

[1:19:04] into something

[1:19:06] um quite different just as

[1:19:10] like if you have a toddler they will

[1:19:11] eventually grow into something quite

[1:19:13] different like an adult is in many

[1:19:15] respects almost a different kind of

[1:19:17] entity than a toddler in terms of what's

[1:19:20] going on inside their minds the kind of

[1:19:21] problems they're dealing with. Um, and

[1:19:25] yet

[1:19:27] for the most part, we don't think it's

[1:19:28] bad for a toddler to grow up, even

[1:19:30] though it means the toddler is no longer

[1:19:32] there. And so perhaps similarly with us

[1:19:35] humans, we now think of us as being

[1:19:37] grown-ups and being so very mature and

[1:19:39] big. But I think we are like babies

[1:19:42] really. Uh, it's just kind of stunted

[1:19:47] because of our biological constraints.

[1:19:49] So we just cease growing up.

[1:19:51] >> Yeah. and have this arrested development

[1:19:53] at age 20. And then we stay hovering for

[1:19:56] a few decades and then just as we have

[1:19:59] started to acquire a little bit of

[1:20:01] knowledge and experience, our brain

[1:20:03] starts to rot and it's all erased again.

[1:20:06] And and maybe that could be a way to

[1:20:07] sort of allow us to continue to grow and

[1:20:09] develop together with with our friends

[1:20:12] and and communities.

[1:20:14] Um, and it would be exciting to see what

[1:20:16] kinds of level of maturity and like

[1:20:21] self-realization that would be possible

[1:20:23] if we could continue, you know, for many

[1:20:25] hundreds of years. And then maybe

[1:20:27] eventually you could imagine adding

[1:20:29] extra neurons and extra memory and extra

[1:20:33] forms of vitality and new emotional

[1:20:36] responses and so forth. I I think we

[1:20:39] just haven't seen nothing yet in terms

[1:20:41] of what's possible. Yeah, I I I totally

[1:20:43] agree with you. There was a science

[1:20:45] fiction book written by Arthur C. Clark

[1:20:48] years ago called Childhood's End.

[1:20:51] Basically, the end of the childhood of

[1:20:53] the species called Humankind, you know,

[1:20:56] and this may well be the moment where

[1:20:59] we're starting on the trajectory of

[1:21:01] ending the childhood of the species and

[1:21:03] it's exciting to see where it goes. All

[1:21:05] the same though, just to paraphrase what

[1:21:06] you said earlier, it's both exciting and

[1:21:08] terrifying and you have to perhaps have

[1:21:11] a fatalistic outlook, do everything that

[1:21:13] we can to shape the best future we could

[1:21:16] have and we'll just see where it all

[1:21:18] goes. So Nick Boston, thank you so much

[1:21:21] for joining us. Uh this is really a

[1:21:23] exciting conversation and uh I look

[1:21:26] forward to seeing where this all goes.

[1:21:28] >> Yeah, it'll be an exciting ride if

[1:21:30] nothing else.

[1:21:31] >> Absolutely. Thank you.

[1:21:33] >> Thanks.

[1:22:03] Heat.

[1:22:14] Heat.