[0:00] In October, over 850 experts, including yourself and other leaders like Richard
[0:04] Branson and Jeffrey Hinton, signed a statement to ban AI super intelligence
[0:08] as you guys raised concerns of potential human extinction. >> Because unless we figure out how do we
[0:14] guarantee that the AI systems are safe, we're toast. >> And you've been so influential on the
[0:19] subject of AI, you wrote the textbook that many of the CEOs who are building
[0:23] some of the AI companies now would have studied on the subject of AI. Yeah.
[0:26] >> So, do you have any regrets? Um, >> Professor Stuart Russell has been named
[0:33] one of Time magazine's most influential voices in AI. >> After spending over 50 years
[0:38] researching, teaching, and finding ways to design >> AI in such a way that
[0:42] >> humans maintain control, >> you talk about this gorilla problem as a
[0:46] way to understand AI in the context of humans. >> Yeah. So, a few million years ago, the
[0:50] human line branched off from the gorilla line in evolution, and now the gorillas
[0:53] have no say in whether they continue to exist because we are much smarter than
[0:57] they are. So intelligence is actually the single most important factor to control planet Earth.
[1:01] >> Yep. >> But we're in the process of making something more intelligent than us. >> Exactly.
[1:05] >> Why don't people stop then? >> Well, one of the reasons is something
[1:08] called the Midas touch. So King Midas is this legendary king who asked the gods,
[1:12] can everything I touch turn to gold? And we think of the Midas touch as being a
[1:15] good thing, but he goes to drink some water, the water has turned to gold. And
[1:19] he goes to comfort his daughter, his daughter turns to gold. So he dies in
[1:22] misery and starvation. So this applies to our current situation in two ways.
[1:26] One is that greed is driving these companies to pursue technology with the
[1:30] probabilities of extinction being worse than playing Russian roulette. And
[1:34] that's even according to the people developing the technology without our
[1:37] permission. And people are just fooling themselves if they think it's naturally
[1:41] going to be controllable. So, you know, after 50 years, I could
[1:45] retire, but instead I'm working 80 or 100 hours a week trying to move things
[1:49] in the right direction. So, if you had a button in front of you which would stop
[1:53] all progress in artificial intelligence, would you press it?
[1:58] >> Not yet. I think there's still a decent chance they guarantee safety. And I can
[2:02] explain more of what that is. >> I see messages all the time in the
[2:08] comments section that some of you didn't realize you didn't subscribe. So, if you
[2:12] could do me a favor and double check if you're a subscriber to this channel,
[2:14] that would be tremendously appreciated. It's the simple, it's the free thing
[2:18] that anybody that watches this show frequently can do to help us here to
[2:21] keep everything going in this show in the trajectory it's on. So, please do
[2:25] double check if you've subscribed and uh thank you so much because in a strange
[2:28] way you are you're part of our history and you're on this journey with us and I
[2:32] appreciate you for that. So, yeah, thank you.
[2:41] Professor Stuart Russell, OBBE. A lot of people have been talking about AI for
[2:46] the last couple of years. It appears you've this really shocked me. It
[2:50] appears you've been talking about AI for most of your life.
[2:53] >> Well, I started doing AI in high school um back in England, but then I did my
[2:59] PhD starting in ' 82 at Stanford. I joined the faculty of Berkeley in ' 86.
[3:06] So I'm in my 40th year as a professor at Berkeley. The main thing that the AI
[3:10] community is familiar with in my work uh is a textbook that I wrote.
[3:16] >> Is this the textbook that most students who study AI are likely learning from? >> Yeah.
[3:24] >> So you wrote the textbook on artificial intelligence 31 years ago. You actually start probably
[3:32] started writing it because it's so bloody big in the year that I was born. So I was born in 92.
[3:36] >> Uh yeah, took me about two years. >> Me and your book are the same age, which just is wonderful
[3:43] way for me to understand just how long you've been talking about this and how
[3:47] long you've been writing about this. And actually, it's interesting that many of
[3:51] the CEOs who are building some of the AI companies now probably learned from your
[3:56] textbook. you had a conversation with somebody who said that in order for
[4:01] people to get the message that we're going to be talking about today, there
[4:05] would have to be a catastrophe for people to wake up. Can you give me
[4:10] context on that conversation and a gist of who you had this conversation with?
[4:14] >> Uh, so it was with one of the CEOs of uh a leading AI company. He sees two
[4:21] possibilities as do I which is um either we have a small or let's say
[4:28] small scale disaster of the same scale as Chernobyl >> the nuclear meltdown in Ukraine.
[4:34] >> Yeah. So this uh nuclear plant blew up in 1986 killed uh a fair number of people directly and
[4:44] maybe tens of thousands of people indirectly through uh radiation. recent
[4:49] cost estimates more than a trillion dollars. So that would wake people up. That would
[4:58] get the governments to regulate. He's talked to the governments and they won't
[5:01] do it. So he looked at this Chernobyl scale disaster as the best case scenario
[5:09] because then the governments would regulate and require AI systems to be
[5:14] built. And is this CEO building an AI company? >> He runs one of the leading AI companies.
[5:22] >> And even he thinks that the only way that people will wake up is if there's a
[5:26] Chernobyl level nuclear disaster. >> Uh yeah, not wouldn't have to be a
[5:29] nuclear disaster. It would be either an AI system that's being misused
[5:35] by someone, for example, to engineer a pandemic or an AI system that does
[5:40] something itself, such as crashing our financial system or our communication
[5:45] systems. The alternative is a much worse disaster where we just lose control
[5:50] altogether. You have had lots of conversations with lots of people in the
[5:54] world of AI, both people that are, you know, have built the technology, have
[5:58] studied and researched the technology or the CEOs and founders that are currently
[6:02] in the AI race. What are some of the the interesting sentiments that the general
[6:07] public wouldn't believe that you hear privately about their perspectives?
[6:14] Because I find that so fascinating. I've had some private conversations with
[6:18] people very close to these tech companies and the shocking sentiment that I was exposed to was that
[6:24] they are aware of the risks often but they don't feel like there's anything
[6:27] that can be done so they're carrying on which is feels like a bit of a paradox to me like
[6:31] >> yes it's it's it must be a very difficult position to be in in a sense right you're you're
[6:38] doing something that you know has a good chance of bringing an end to life on
[6:44] including that of yourself and your own family. They feel that they can't escape this race, right?
[6:54] If they, you know, if a CEO of one of those companies was to say, you know, we're
[6:59] we're not going to do this anymore, they would just be replaced
[7:04] because the investors are putting their money up because they want to create AGI
[7:10] and reap the benefits of it. So, it's a strange situation where every at least
[7:16] all the ones I've spoken to, I haven't spoken to Sam Wolman about this, but you know, Sam Wolman
[7:23] even before becoming CEO of Open AI said that creating superhuman intelligence is the
[7:32] biggest risk to human existence that there is. My worst fears are that we
[7:38] cause significant we the field the technology the industry cause significant harm to the world.
[7:43] >> You know Elon Musk is also on record saying this. So uh Dario Ammedday
[7:48] estimates up to a 25% risk of extinction. >> Was there a particular moment when you realized that
[7:56] the CEOs are well aware of the extinction level risks? I mean, they all
[8:01] signed a statement in May of 23 uh called it's called the extinction
[8:07] statement. It basically says AGI is an extinction risk at the same level as
[8:12] nuclear war and pandemics. But I don't think they feel it in their
[8:17] gut. You know, imagine that you were one of the nuclear physicists. You know, I
[8:24] guess you've seen Oppenheimer, right? you're there, you're watching that first nuclear explosion.
[8:30] How how would that make you feel about the potential impact of nuclear war on
[8:37] the human race? Right? I I think you would probably become a pacifist and say
[8:43] this weapon is so terrible, we have got to find a way to uh keep it under
[8:49] control. We are not there yet with the people making these decisions
[8:55] and certainly not with the governments, right? You know what policy makers do is they, you know,
[9:03] they listen to experts. They keep their finger in the wind. You got some
[9:09] experts, you know, dangling $50 billion checks and saying, "Oh, you know, all
[9:15] that doomer stuff, it's just fringe nonsense. don't worry about it. Take my
[9:19] $50 billion check. You know, on the other side, you've got very
[9:23] well-meaning, brilliant scientists like like Jeff Hinton saying, actually, no,
[9:28] this is the end of the human race. But Jeff doesn't have a $50 billion check.
[9:34] So the view is the only way to stop the race is if governments intervene
[9:40] and say okay we don't we don't want this race to go ahead until we can be sure
[9:47] that it's going ahead in absolute safety. >> Closing off on your career journey, you
[9:55] got a you received an OB from Queen Elizabeth. >> Uh yes. >> And what was the listed reason for that
[10:00] for the award? uh contributions to artificial intelligence research
[10:05] >> and you've been listed as a Time magazine most influential person in in
[10:10] AI several years in a row including this year in 2025. >> Y >> now there's two terms here that are
[10:18] central to the things we're going to discuss. One of them is AI and the other is AGI.
[10:22] In my muggle interpretation of that, it's artificial general intelligence is
[10:27] when the system, the computer, whatever it might be, the technology has
[10:31] generalized intelligence, which means that it could theoretically see, understand
[10:37] um the world. It knows everything. It can understand everything in the the
[10:42] world as well as or better than a human being. >> Y >> can do it.
[10:46] >> And I think take action as well. I mean some some people say oh you know AGI
[10:51] doesn't have to have a body but a good chunk of our intelligence actually is
[10:56] about managing our body about perceiving the real environment and acting on it
[11:01] moving grasping and so on. So I think that's part of intelligence and and AGI
[11:07] systems should be able to operate robots successfully. But there's often a misunderstanding,
[11:13] right, that people say, well, if it doesn't have a robot body, then it can't
[11:17] actually do anything. But then if you remember, most of us don't do things with our bodies.
[11:25] Some people do, brick layers, painters, gardeners, chefs, um, but people who do podcasts,
[11:35] you're doing it with your mind, right? you're doing it with your ability to to
[11:40] produce language. Uh, you know, Adolf Hitler didn't do it with his body.
[11:46] He did it by producing language. >> Hope you're not comparing us. But
[11:54] but uh you know so even an AGI that has no body uh it actually has more access
[12:01] to the human race than Adolf Hitler ever did because it can send emails and texts to
[12:10] what threearters of the world's population directly. It can it also speaks all of their languages
[12:17] and it can devote 24 hours a day to each individual person on earth to convince
[12:24] them of to do whatever it wants them to do. >> And our whole society runs now on the
[12:28] internet. I mean if there's an issue with the internet, everything breaks
[12:31] down in society. Airplanes become grounded and we'll have electricity is
[12:35] running off as internet systems. So I mean my entire life it seems to run off the internet now.
[12:42] >> Yeah. water supplies. So, so this is one of the roots by which AI systems could
[12:48] bring about a medium-sized catastrophe is by basically shutting down our life support systems.
[12:58] >> Do you believe that at some point in the coming decades we'll arrive at a point
[13:04] of AGI where these systems are generally intelligent? Uh yes, I think it's virtually certain
[13:12] unless something else intervenes like a nuclear war or or we may refrain from
[13:19] doing it. But I think it will be extraordinarily difficult uh for us to refrain.
[13:25] >> When I look down the list of predictions from the top 10 AI CEOs on when AGI will
[13:30] arrive, you've got Sam Alman who's the founder of OpenAI/ChatGBT
[13:35] um says before 2030. Demis at DeepMind says 2030 to 2035. Jensen from Nvidia says around five
[13:46] years. Daario at Anthropic says 2026 to 2027. Powerful AI close to AGI. Elon
[13:53] says in the 2020s. Um and go down the list of all of them and they're all
[13:58] saying relatively within 5 years. >> I actually think it'll take longer. I
[14:03] don't think you can make a prediction based on engineering um in the sense that yes, we could make
[14:14] machines 10 times bigger and 10 times faster, but that's probably not the reason why
[14:20] we don't have AGI, right? In fact, I think we have far more computing power
[14:27] than we need for AGI. maybe a thousand times more than we need. The reason we
[14:34] don't have AGI is because we don't understand how to make it properly. Um what we've seized upon
[14:42] is one particular technology called the language model. And we observed that as
[14:49] you make language models bigger, they produce text language that's more
[14:55] coherent and sounds more intelligent. And so mostly what's been happening in
[15:01] the last few years is just okay let's keep doing that because one thing
[15:06] companies are very good at unlike universities is spending money. They
[15:11] have spent gargantuan amounts of money and they're going to spend even more
[15:17] gargantuan amounts of money. I mean you know we mentioned nuclear weapons. So the Manhattan project
[15:24] uh in World War II to develop nuclear weapons, its budget in 2025
[15:32] was about 20 odd billion dollars. The budget for AGI is going to be a trillion
[15:41] dollars next year. So 50 times bigger than the Manhattan project. Humans have
[15:46] a remarkable history of figuring things out when they galvanize towards a shared objective.
[15:53] You know, thinking about the moon landings or whatever it else it might be
[15:57] through history. And the thing that makes this feel all quite inevitable to
[16:01] me is just the sheer volume of money being invested into it. I've never seen
[16:05] anything like it in my life. >> Well, there's never been anything like
[16:07] this in history. Is this the biggest technology project in human history by
[16:12] orders of magnitude? And there doesn't seem to be anybody that is pausing to ask the questions
[16:20] about safety. It doesn't it doesn't even appear that there's room for that in
[16:23] such a race. I think that's right. To varying extents, each of these companies
[16:29] has a division that focuses on safety. Does that division have any sway? Can
[16:35] they tell the other divisions, no, you can't release that system? Not really. Um
[16:42] I think some of the companies do take it more seriously. Anthropic
[16:47] uh does. I think Google DeepMind even there I think the commercial imperative
[16:54] to be at the forefront is absolutely vital. If a company is perceived as
[17:03] you know falling behind and not likely to be competitive, not likely to be the
[17:09] one to reach AGI first, then people will move their money elsewhere very quickly.
[17:16] >> And we saw some quite high-profile departures from company like companies
[17:19] like OpenAI. Um, I know a chap called Yan Leak left who was working on AI
[17:27] safety at OpenAI and he said that the reason for his leaving was that safety
[17:32] culture and processes processes have taken a backseat to shiny products at
[17:36] OpenAI and he gradually lost trust in leadership but also Ilia Sutskysa
[17:42] >> Ilia Sutska yeah so he was the >> co-founder co-founder and chief scientist for a while and then
[17:48] >> yeah so he and Yan Lea are the main safety people. Um, and so when they say OpenAI doesn't care
[17:58] about safety, that's pretty concerning. >> I've heard you talk about this gorilla problem.
[18:06] What is the gorilla problem as a way to understand AI in the context of humans?
[18:11] >> So, so the gorilla problem is is the problem that gorillas face with respect to humans.
[18:19] So you can imagine that you know a few million years ago the the human line
[18:23] branched off from the gorilla line in evolution. Uh and now the gorillas are
[18:28] looking at the human line and saying yeah was that a good idea
[18:33] and they have no um they have no say in whether they continue to exist
[18:39] >> because we have a we are much smarter than they are. if we chose to, we could
[18:43] make them extinct in in a couple of weeks and there's nothing they can do about it.
[18:50] So that's the gorilla problem, right? Just the the problem a species faces
[18:56] when there's another species that's much more capable. >> And so this says that intelligence is
[19:02] actually the single most important factor to control planet Earth. Yes.
[19:06] Intelligence is the ability to bring about what you want in the world.
[19:12] >> And we're in the process of making something more intelligent than us. >> Exactly.
[19:16] >> Which suggests that maybe we become the gorillas. >> Exactly. Yeah.
[19:21] >> Is that is there any fault in the reasoning there? Because it seems to
[19:24] make such perfect sense to me. But if it Why doesn't Why don't people stop
[19:30] then? cuz it it seems like a crazy thing to want to >> because they think that uh if they
[19:37] create this technology, it will have enormous economic value. They'll be able
[19:42] to use it to replace all the human workers in the world uh to develop new uh products, drugs,
[19:52] um forms of entertainment, any anything that has economic value, you could use
[19:57] AGI to to create it. And and maybe it's just an irresistible thing in itself,
[20:04] right? I think we as humans place so much store on our intelligence. You know, you know, how we
[20:15] think about, you know, what is the pinnacle of human achievement?
[20:19] If we had AGI, we could go way higher than that. So it it's very seductive for
[20:27] people to want to create this technology and I think people are just fooling
[20:34] themselves if they think it's naturally going to be controllable. I mean the question is
[20:43] how are you going to retain power forever over entities more powerful than yourself?
[20:50] >> Pull the plug out. People say that sometimes in the comment section when we
[20:54] talk about AI, they said, "Well, I'll just pull a plug out."
[20:56] >> Yeah, it's it's sort of funny. In fact, you know, yeah, reading the comment
[20:59] sections in newspapers, whenever there's an AI article, there'll be people who say, "Oh, you can
[21:07] just pull the plug out, right?" As if a super intelligent machine would never
[21:10] have thought of that one. Don't forget who's watched all those films where they
[21:14] did try to pull the plug out. Another thing they said, well, you know, as long
[21:17] as it's not conscious, then it doesn't matter. It won't ever do anything. Um, which is
[21:29] completely off the point because, you know, I I don't think the gorillas are
[21:34] sitting there saying, "Oh, yeah, you know, if only those humans hadn't been
[21:38] conscious, everything would have be fine, >> right?" No, of course not. What would
[21:43] make gorillas go extinct is the things that humans do, right? How we behave,
[21:48] our ability to act successfully in the world. So when I play chess
[21:54] against my iPhone and I lose, right, I don't I don't think, oh, well, I'm
[22:01] losing because it's conscious, right? No, I'm just losing because it's better
[22:04] than I am at at in that little world uh moving the bits around uh to to get what
[22:10] it wants. and and so consciousness has nothing to do with it, right? Competence
[22:16] is the thing we're concerned about. So I think the only hope is can we simultaneously
[22:25] build machines that are more intelligent than us but guarantee that they will always act in our best
[22:35] interest. So throwing that question to you, can we build machines that are more intelligent
[22:40] than us that will also always act in our best interests? It sounds like a bit of a uh
[22:46] contradiction to some degree because it's kind of like me saying I've got a
[22:51] French bulldog called Pablo that's uh 9 years old >> and it's like saying that he could be
[22:57] more intelligent than me yet I still walk him and decide when he gets fed. I
[23:02] think if he was more intelligent than me he would be walking me. I'd be on the leash.
[23:06] >> That's the That's the trick, right? Can we make AI systems whose only purpose is
[23:12] to further human interests? And I think the answer is yes. And this is actually what I've been
[23:19] working on. So I I think one part of my career that I didn't mention is is sort
[23:25] of having this epiphany uh while I was on sabbatical in Paris. This was 2013 or
[23:32] so. just realizing that further progress in the capabilities of AI uh you know if if we succeeded in
[23:43] creating real superhuman intelligence that it was potentially a catastrophe
[23:49] and so I pretty much switched my focus to work on how do we make it so that
[23:55] it's guaranteed to be safe. Are you somewhat troubled by everything that's going on at the moment
[24:02] with with AI and how it's progressing? Because you strike me as someone that's
[24:08] somewhat troubled under the surface by the way things are moving forward and
[24:14] the speed in which they're moving forward. >> That's an understatement. I'm appalled
[24:20] actually by the lack of attention to safety. I mean, imagine if someone's
[24:26] building a nuclear power station in your neighborhood and you go along to the chief engineer
[24:33] and you say, "Okay, these nuclear thing, I've heard that they can actually
[24:38] explode, right? There was this nuclear explosion that happened in Hiroshima, so
[24:43] I'm a bit worried about this. You know, what steps are you taking to make sure
[24:47] that we don't have a nuclear explosion in our backyard?" And the chief engineer says, "Well, we
[24:54] thought about it. We don't really have an answer." >> Yeah. >> You would, what would you say?
[25:03] I think you would you would use some exploitives. >> Well, >> and you'd call your MP and say, you
[25:11] know, get these people out. >> I mean, what are they doing? You read out the list of you know
[25:20] projected dates for AGI but notice also that those people I think I mentioned Darday says a 25%
[25:28] chance of extinction. Elon Musk has a 30% chance of extinction. Sam Alolman says
[25:36] basically that AGI is the biggest risk to human existence. So what are they doing? They are playing
[25:42] Russian roulette with every human being on Earth. without our permission. They're coming
[25:48] into our houses, putting a gun to the head of our children, pulling the trigger, and saying, "Well,
[25:56] you know, possibly everyone will die. Oops. But possibly we'll get incredibly rich."
[26:04] That's what they're doing. Did they ask us? No. Why is the government allowing them to do this?
[26:12] because they dangle $50 billion checks in front of the governments.
[26:17] So I think troubled under the surface is an understatement. >> What would be an accurate statement?
[26:24] >> Appalled and I I am devoting my life to trying to divert from this course of history
[26:34] into a different one. Do you have any regrets about things you could have done in the past because
[26:40] you've been so influential on the subject of AI? You wrote the textbook
[26:44] that many of these people would have studied on the subject of AI more than
[26:47] 30 years ago. Do do you have when you're alone at night and you think about
[26:50] decisions you've made on this in this field because of your scope of
[26:53] influence? Is there anything you you regret? >> Well, I do wish I had understood
[26:59] earlier uh what I understand now. we could have developed safe AI systems. I think the there are
[27:08] some weaknesses in the framework which I can explain but I think that framework
[27:12] could have evolved to develop actually safe AI systems where we could prove
[27:18] mathematically that the system is going to act in our interests. The kind of AI
[27:24] systems we're building now, we don't understand how they work.
[27:28] >> We don't understand how they work. It's it's a strange thing to build something
[27:33] where you don't understand how it works. I mean, there's no sort of comparable
[27:36] through human history. Usually with machines, you can pull it apart and see
[27:39] what cogs are doing what and how the >> Well, actually, we we put the cogs
[27:43] together, right? So, with with most machines, we designed it to have a
[27:48] certain behavior. So, we don't need to pull it apart and see what the cogs are
[27:51] because we put the cogs in there in the first place, right? one by one we
[27:55] figured out what what the pieces needed to be how they work together to produce
[27:59] the effect that we want. So the best analogy I can come up with is you know
[28:06] the the first cave person who left a bowl of fruit in the sun and forgot
[28:12] about it and then came back a few weeks later and there was sort of this big
[28:16] soupy thing and they drank it and got completely shitfaced. >> They got drunk. Okay.
[28:21] >> And they got this effect. They had no idea how it worked, but they were very
[28:26] happy about it. And no doubt that person made a lot of money from it.
[28:31] >> Uh so yeah, it it is kind of bizarre, but my mental picture of these things is
[28:36] is like a chain link fence, right? So you've got lots of these connections
[28:43] and each of those connections can be its connection strength can be adjusted
[28:48] and then uh you know a signal comes in one end of this chain link fence and
[28:54] passes through all these connections and comes out the other end and the signal
[28:59] that comes out the other end is affected by your adjusting of all the connection
[29:03] strengths. So what you do is you you get a whole lot of training data and you
[29:08] adjust all those connection strengths so that the signal that comes out the other
[29:11] end of the network is the right answer to the question. So if your training
[29:16] data is lots of photographs of animals, then all those pixels go in one end of
[29:23] the network and out the other end, you know, it activates the llama output or
[29:30] the dog output or the cat output or the ostrich output. And uh and so you just
[29:35] keep adjusting all the connection strengths in this network until the
[29:38] outputs of the network are the ones you want. >> But we don't really know what's going on
[29:42] across all of those different chains. So what's going on inside that network?
[29:46] Well, so now you have to imagine that this network, this chain link fence is
[29:52] is a thousand square miles in extent. >> Okay, >> so it's covering the whole of the San
[29:58] Francisco Bay area or the whole of London inside the M25, right? That's how big it is.
[30:04] >> And the lights are off. It's night time. So you might have in that network about a trillion
[30:11] uh adjustable parameters and then you do quintilions or sexillions of small
[30:16] random adjustments to those parameters uh until you get the behavior that you
[30:23] want. I've heard Sam Alman say that in the future he doesn't believe they'll
[30:28] need much training data at all to make these models progress themselves because
[30:32] there comes a point where the models are so smart that they can train themselves
[30:37] and improve themselves without us needing to pump in articles and books and scour the internet.
[30:45] >> Yeah, it should it should work that way. So I think what he's referring to and
[30:49] this is something that several companies are now worried might start happening
[30:56] is that the AI system becomes capable of doing AI research by itself.
[31:05] And so uh you have a system with a certain capability. I mean crudely we
[31:10] could call it an IQ but it's it's not really an IQ. But anyway, imagine that
[31:16] it's got an IQ of 150 and uses that to do AI research, comes up with better algorithms or
[31:23] better designs for hardware or better ways to use the data, updates itself. Now it has an IQ of 170,
[31:31] and now it does more AI research, except that now it's got an IQ of 170, so it's
[31:36] even better at doing the AI research. And so, you know, next iteration it's
[31:41] 250 and uh and so on. So this this is an idea that one of Alan Turing's friends
[31:48] good uh wrote out in 1965 called the intelligence explosion right that one of
[31:54] the things an intelligence system could do is to do AI research and therefore
[32:00] make itself more intelligent and this would uh this would very rapidly take
[32:05] off and leave the humans far behind. >> Is that what they call the fast takeoff?
[32:10] >> That's called the fast takeoff. Sam Alman said, "I think a fast takeoff is
[32:15] more possible than I thought a couple of years ago." Which I guess is that moment
[32:18] where the AGI starts teaching itself. >> In and in his blog, the gentle
[32:22] singularity, he said, "We may already be past the event horizon of takeoff."
[32:29] >> And what does what does he mean by event horizon? The event horizon is is a
[32:33] phrase borrowed from astrophysics and it refers to uh the black hole. And the
[32:40] event horizon, think it if you got some very very massive object that's heavy
[32:46] enough that it actually prevents light from escaping. That's why it's called
[32:51] the black hole. It's so heavy that light can't escape. So if you're inside the
[32:56] event horizon then then light can't escape beyond that. So I think what he's
[33:03] what he's meaning is if we're beyond the event horizon it means that you know now
[33:07] we're just trapped in the gravitational attraction of the black hole or in this case we're
[33:15] we're trapped in the inevitable slide if you want towards AGI. When you when you think about the
[33:23] economic value of AGI, which I've estimated at uh 15 quadrillion dollars,
[33:30] that acts as a giant magnet in the future. >> We're being pulled towards it.
[33:35] >> We're being pulled towards it. And the closer we get, the stronger the force,
[33:41] the probability, you know, the closer we get, the the the higher the probability
[33:44] that we will actually get there. So, people are more willing to invest. And
[33:49] we also start to see spin-offs from that investment such as chat GBT, right, which is, you
[33:56] know, generates a certain amount of revenue and so on. So, so it does act as
[34:01] a magnet and the closer we get, the harder it is to pull out of that field.
[34:07] >> It's interesting when you think that this could be the the end of the human
[34:10] story. this idea that the end of the human story was that we created our
[34:15] successor like we we summoned our next iteration of life or intelligence ourselves like we
[34:25] took ourselves out. It is quite like just removing ourselves and the
[34:29] catastrophe from it for a second. It is it is an unbelievable story.
[34:34] >> Yeah. And you know there are many legends the sort of be careful what you wish for
[34:43] legend and in fact the king Midas legend is is very relevant here. >> What's that?
[34:49] >> So King Midas is this legendary king who lived in modern day Turkey but I think
[34:56] is sort of like Greek mythology. He is said to have asked the gods to grant him a wish.
[35:04] The wish being that everything I touch should turn to gold. So he's incredibly greedy. Uh you know
[35:12] we call this the mightest touch. And we think of the mightest touch as being
[35:16] like you know that's a good thing, right? Wouldn't that be cool? But what
[35:20] happens? So he uh you know he goes to drink some water and he finds that the
[35:25] water has turned to gold. And he goes to eat an apple and the apple turns to
[35:29] gold. and he goes to you know comfort his daughter and his daughter turns to gold
[35:35] and so he dies in misery and starvation. So this applies to our current situation
[35:42] in in two ways actually. So one is that I think greed is driving us to pursue
[35:51] a technology that will end up consuming us and we will perhaps die in misery and
[35:57] starvation instead. The what it shows is how difficult it is to correctly
[36:04] articulate what you want the future to be like. For a long time, the way we
[36:11] built AI systems was we created these algorithms where we could specify the
[36:16] objective and then the machine would figure out how to achieve the objective
[36:20] and then achieve it. So, you know, we specify what it means to win at chess or
[36:25] to win at go and the algorithm figures out how to do it uh and it does it
[36:29] really well. So that was, you know, standard AI up until recently. And it
[36:34] suffers from this drawback that sure we know how to specify the objective in
[36:38] chess, but how do you specify the objective in life, right? What do we
[36:43] want the future to be like? Well, really hard to say. And almost any attempt to
[36:48] write it down precisely enough for the machine to bring it about would be
[36:53] wrong. And if you're giving a machine an objective which isn't aligned with what
[36:58] we truly want the future to be like, right, you're actually setting up a
[37:02] chess match and that match is one that you're going to lose when the machine is
[37:07] sufficiently intelligent. And so that that's that's problem number one.
[37:12] Problem number two is that the kind of technology we're building now, we don't
[37:16] even know what its objectives are. So it's not that we're specifying the
[37:21] objectives, but we're getting them wrong. We're growing these systems. They have objectives,
[37:28] but we don't even know what they are because we didn't specify them. What
[37:31] we're finding through experiment with them is that they seem to have an extremely strong
[37:37] self-preservation objective. >> What do you mean by that? >> You can put them in hypothetical
[37:41] situations. either they're going to get switched off and replaced or they have
[37:48] to allow someone, let's say, you know, someone has been locked in a machine
[37:52] room that's kept at 3 centigrades or they're going to freeze to death.
[37:58] They will choose to leave that guy locked in the machine room and die rather than be switched off
[38:03] themselves. >> Someone's done that test. >> Yeah. >> What was the test? They they asked they
[38:10] asked the AI. >> Yep. They put well they put them in these hypothetical situations and they
[38:14] allow the AI to decide what to do and it decides to preserve its own existence,
[38:19] let the guy die and then lie about it. In the King Midas analogy story, one of
[38:27] the things that highlights for me is that there's always trade-offs in life
[38:30] generally. And you know, especially when there's great upside, there always
[38:34] appears to be a pretty grave downside. Like there's almost nothing in my life
[38:37] where I go, it's all upside. Like even like having a dog, it shits on my
[38:41] carpet. My girlfriend, you know, I love her, but you know, not always easy. Even
[38:47] with like going to the gym, I have to pick up these really, really heavy
[38:49] weights at 10 p.m. at night sometimes when I don't feel like it. There's
[38:53] always to get the muscles or the six-pack. There's always a trade-off.
[38:56] And when you interview people for a living like I do, >> you know, you hear about so many
[38:59] incredible things that can help you in so many ways, but there is always a
[39:03] trade-off. There's always a way to overdo it. Mhm. >> Melatonin will help you sleep, but it
[39:07] will also you'll wake up groggy and if you overdo it, your brain might stop
[39:11] making melatonin. Like I can go through the entire list and one of the things
[39:13] I've always come to learn from doing this podcast is whenever someone
[39:17] promises me a huge upside for something, it'll cure cancer. It'll be a utopia.
[39:21] You'll never have to work. You'll have a butler around your house.
[39:24] >> I my my first instinct now is to say, at what cost? >> Yeah.
[39:27] >> And when I think about the economic cost here, if we start if we start there,
[39:32] >> have you got kids? >> I have four. Yeah. >> Four kids. What what how old is the youngest kid
[39:37] that you 19? >> 19. Okay. So your if you say your kids were were 10 now
[39:42] >> and they were coming to you and they're saying, "Dad, what do you think I should study
[39:46] >> based on the way that you see the future? >> A future of AGI, say if all these CEOs
[39:52] are right and they're predicting AGI within 5 years, what should I study, Dad?"
[39:57] >> Well, okay. So let's look on the bright side and say that the CEOs all decide to
[40:03] pause their AGI development, figure out how to make it safe and then resume uh
[40:09] in whatever technology path is actually going to be safe. What does that do to human life
[40:14] >> if they pause? >> No. If if they succeed in creating AGI and they solve the safety problem
[40:21] >> and they solve the safety problem. Okay. Yeah. Cuz if they don't solve the safety
[40:24] problem, then you know, you should probably be finding a bunker or
[40:30] going to Patagonia or somewhere in New Zealand. >> Do you mean that? Do you think I should
[40:33] be finding a bunker if they >> No, because it's not actually going to
[40:35] help. Uh, you know, it's not as if the AI system couldn't find you or I mean,
[40:40] it's interesting. So, we're going off on a little bit of a digression here
[40:44] >> for from your question, but I'll come back to it. >> So, people often ask, well, okay, so how
[40:49] exactly do we go extinct? And of course, if you ask the gorillas or the dodos,
[40:54] you know, how exactly do you think you're going to go extinct?
[40:58] They have the faintest idea. Humans do something and then we're all dead. So,
[41:02] the only things we can imagine are the things we know how to do that might
[41:06] bring about our own extinction, like creating some carefully engineered
[41:11] pathogen that infects everybody and then kills us or starting a nuclear war.
[41:17] presumably is something that's much more intelligent than us would have much
[41:21] greater control over physics than we do. And we already do amazing things, right?
[41:27] I mean, it's amazing that I can take a little rectangular thing out of my
[41:31] pocket and talk to someone on the other side of the world or even someone in
[41:35] space. It's just astonishing and we take it for granted, right? But imagine you
[41:41] know super intelligent beings and their ability to control physics you know
[41:45] perhaps they will find a way to just divert the sun's energy sort of go
[41:50] around the earth's orbit so you know literally the earth turns into a snowball in in a few days
[41:56] >> maybe they'll just decide to leave >> leave leave the earth maybe they'd look
[42:01] at the earth and go this isn't this is not interesting we know that over there
[42:04] there's an even more interesting planet we're going to go over there and they
[42:07] just I don't know get on a rocket or teleport themselves They might. Yeah.
[42:11] So, it's it's difficult to anticipate all the ways that we might go extinct at the hands of
[42:17] entities much more intelligent than ourselves. Anyway, coming back to the
[42:23] question of well, if everything goes right, right, if we we create AGI, we
[42:27] figure out how to make it safe, we we achieve all these economic miracles,
[42:32] then you face a problem. And this is not a new problem, right? So, so John
[42:36] Maynard Kanes who was a famous economist in the early part of the 20th century
[42:40] wrote a wrote a paper in 1930. So, this is in the depths of the
[42:44] depression. It's called on the economic problems of our grandchildren. He
[42:49] predicts that at some point science will will deliver sufficient wealth that no
[42:55] one will have to work ever again. And then man will be faced with his true eternal problem.
[43:02] How to live? I don't remember the exact word but how to live wisely and well
[43:07] when the you know the economic incentives the economic constraints are
[43:12] lifted we don't have an answer to that question right so AI systems are doing
[43:18] pretty much everything we currently call work anything you might aspire to like you
[43:23] want to become a surgeon it takes the robot seven seconds to learn how to be a surgeon that's better
[43:29] than any human being >> Elon said last week that The humanoid
[43:33] robots will be 10 times better than any surgeon that's ever lived.
[43:37] >> Quite possibly. Yeah. Well, and they'll also have, you know, h they'll have
[43:42] hands that are, you know, a millimeter in size, so they can go inside and do
[43:46] all kinds of things that humans can't do. And I think we need to put serious
[43:51] effort into this question. What is a world where AI can do all forms of human
[43:58] work that you would want your children to live in? What does that world look like? Tell me
[44:04] the destination so that we can develop a transition plan to get there. And I've asked AI
[44:11] researchers, economists, science fiction writers, futurists, no one has been able
[44:18] to describe that world. I'm not saying it's not possible. I'm just saying I've
[44:22] asked hundreds of people in multiple workshops. It does not, as far as I
[44:27] know, exist in science fiction. You know, it's notoriously difficult to
[44:32] write about a utopia. It's very hard to have a plot, right? Nothing bad happens
[44:37] in in utopia. So, it's difficult to make a plot. So, usually you start out with a
[44:42] utopia and then it all falls apart and that's how that's how you get get a
[44:46] plot. You know that there's one series of novels people point to where humans
[44:51] and super intelligent AI systems coexist. It's called The Culture Novels
[44:56] by Ian Banks. highly recommended for those people who like science fiction
[45:02] and and they absolutely the AI systems are only concerned with furthering human
[45:08] interests. They find humans a bit boring and but nonetheless they they are there
[45:12] to help. But the problem is you know in that world there's still nothing to do
[45:18] to find purpose. In fact, you know, the the subgroup of humanity that has
[45:23] purpose is the subgroup whose job it is to expand the boundaries of our galactic
[45:29] civilization. Some cases fighting wars against alien species and and so on,
[45:35] right? So that's the sort of cutting edge and that's 0.01% of the population.
[45:41] Everyone else is desperately trying to get into that group so they have some
[45:45] purpose in life. When I speak to very successful billionaires privately off
[45:50] camera, off microphone about this, they say to me that they're investing really
[45:53] heavily in entertainment things like football clubs. Um because people are
[45:59] going to have so much free time that they're not going to know what to do
[46:01] with it and they're going to need things to spend it on. This is what I hear a
[46:05] lot. I've heard this three or four times. I've actually heard Sam Orman say a version of this
[46:09] >> um about the amount of free time we're going to have. I've obviously also heard
[46:12] recently Elon talking about the age of abundance when he delivered his
[46:16] quarterly earnings just a couple of weeks ago and he said that there will be
[46:20] at some point 10 billion humanoid robots. His pay packet um targets him to
[46:25] deliver one 1 million of these human humanoid robots a year that are enabled by AI by 2030.
[46:33] So if he if he does that he gets I think it's part of his package he gets a trillion dollars
[46:38] >> in in compensation. >> Yeah. So the age of abundance for Elon.
[46:43] It's not that it's absolutely impossible to have a worthwhile world of that, you
[46:50] know, with that premise, but I'm just waiting for someone to describe it.
[46:54] >> Well, maybe. So, let me try and describe it. Uh, we wake up in the morning, we go
[47:01] and watch some form of human centric entertainment or participate in some form of human
[47:08] centric entertainment. Mhm. >> We we go to retreats and with each other
[47:14] and sit around and talk about stuff. >> Mhm. >> And maybe people still listen to podcasts. >> Okay.
[47:24] >> I hope I hope so for your sake. >> Yeah. Um it it feels a little bit like a cruise ship
[47:33] and you know and there are some cruises where you know it's smarty bands people
[47:37] and they have you know they have lectures in the evening about ancient
[47:41] civilizations and whatnot and some are more uh more popular entertainment and
[47:46] this is in fact if you've seen the film Walle this is one picture of that future in fact in Wle
[47:55] the human race are all living on cruise ships in space. They have no
[48:00] constructive role in their society, right? They're just there to consume
[48:04] entertainment. There's no particular purpose to education. Uh, you know, and
[48:08] they're depicted actually as huge obese babies. They're actually wearing onesies
[48:15] to emphasize the fact that they have become infeebled. and they become
[48:19] infeeble because there's there's no purpose in being able to do anything at
[48:25] least in in this conception. You know, Wally is not the future that we want.
[48:31] >> Do you think much about humanoid robots and how they're a protagonist in this story of AI?
[48:37] >> It's an interesting question, right? Why why humanoid? And the one of the reasons
[48:43] I think is because in all the science fiction movies, they're humanoid. So
[48:46] that's what robots are supposed to be, right? because they were in science
[48:49] fiction before they became a reality. Right? So even Metropolis which is a
[48:53] film from 1920 I think the robots are humanoid right basically people covered
[48:59] in metal. You know from a practical point of view as we have discovered
[49:04] humanoid is a terrible design because they fall over. Um and uh you know you do want multi-fingered
[49:15] hands of some kind. It doesn't have to be a hand, but you want to have, you
[49:20] know, at least half a dozen appendages that can grasp and manipulate things.
[49:25] And you need something, you know, some kind of locomotion. And wheels are
[49:30] great, except they don't go upstairs and over curbs and things like that. So,
[49:35] that's probably why we're going to be stuck with legs. But a four-legged,
[49:39] twoarmed robot would be much more practical. I guess the argument I've
[49:44] heard is because we've built a human world. So everything the physical spaces
[49:48] we navigate, whether it's factories or our homes or the street or other sort of
[49:54] public spaces are all designed for exactly this physical form. So if we are going to
[50:01] >> to some extent, yeah, but I mean our dogs manage perfectly well to navigate
[50:06] around our houses and streets and so on. So if you had a a centaur,
[50:11] uh it could also navigate, but it can, you know, it can carry much greater
[50:16] loads because it's quadripeda. It's much more stable. If it needs to drive a car,
[50:21] it can fold up two of its legs and and so on so forth. So I think the arguments
[50:25] for why it has to be exactly humanoid are sort of post hawk justification. I
[50:31] think there's much more, well, that's what it's like in the movies and that's
[50:34] spooky and cool, so we need to have them be human. I I don't think it's a good engineering argument.
[50:40] >> I think there's also probably an argument that we would be more accepting of them
[50:46] moving through our physical environments if they represented our form a bit more.
[50:52] Um, I also I was thinking of a bloody baby gate. You know those like
[50:55] kindergarten gates they get on stairs? >> Yeah. >> My dog can't open that. But a humanoid
[51:00] robot could reach over the other side. >> Yeah. And so could a centaur robot,
[51:04] right? So in some sense, centaur robot is >> there's something ghastly about the look
[51:08] of those though. >> Is a humanoid. Well, >> do you know what I mean? Like a
[51:11] four-legged big monster sort of crawling through my house when I have guests over.
[51:15] >> Your dog is a your dog is a four-legged monster. >> I know. Uh so I think actually I I would
[51:22] argue the opposite that um we want a distinct form because they are distinct entities
[51:31] and the more humanoid the worse it is in terms of confusing our subconscious
[51:39] psychological systems. So, I'm arguing from the perspective of the people
[51:43] making them. As in, if I was making the decision whether it to be some
[51:46] four-legged thing that I've that I'm unfamiliar with that I'm less likely to
[51:50] build a relationship with or allow to take care of, I don't know, might might
[51:57] look after my children. Obviously, I'm listen, I'm not saying I would allow
[51:59] this to look after my children, >> but I'm saying from a if I'm building a company,
[52:03] >> the manufacturer would certainly >> Yeah. want want to be >> Yeah. So, I that's an interesting
[52:07] question. I mean there's also what's called the uncanny valley which is a a
[52:13] phrase from computer graphics when they started to make characters in computer
[52:20] graphics they tried to make them look more human right so if you if you for
[52:24] example if you look at Toy Story they're not very humanl looking right if
[52:30] you look at the Incredibles they're not very humanl looking and so we think of
[52:33] them as cartoon characters if you try to make them more human they naturally become repulsive
[52:39] >> until they don't >> until they become very you have to be very very close to perfect in order not
[52:46] to be repulsive. So the the uncanny valley is this I you know like the the
[52:50] gap between you so perfectly human and not at all human but in between it's
[52:54] really awful and uh and so they there were a couple of movies that tried like
[52:59] Polar Express was one where they tried to have quite humanlooking characters
[53:05] you know being humans not not being superheroes or anything else and it's
[53:08] repulsive to watch. I when I watched that shareholder presentation the other
[53:13] day, Elon had these two humanoid robots dancing on stage and I've seen lots of
[53:17] humanoid robot demonstrations over the years. You know, you've seen like the
[53:19] Boston Dynamics dog thing jumping around and whatever else. >> But there was a moment where my brain
[53:26] for the first time ever genuinely thought there was a human in a suit. Mhm.
[53:31] >> And I actually had to research to check if that was really their Optimus robot
[53:34] because the way it was dancing was so unbelievably fluid that for the first
[53:39] time ever, my my my brain has only ever associated those movements with human
[53:45] movements. And I I'll play it on the screen if anyone hasn't seen it, but
[53:48] it's just the robots dancing on stage. And I was like, that is a human in a
[53:52] suit. And it was really the knees that gave it away because the knees were all
[53:55] metal. Huh. I thought there's no way that could be a human knee in a in one
[53:59] of those suits. And he, you know, he says they're going into production next
[54:03] year. They're used internally at Tesla now, but he says they're going into
[54:05] production next year. And it's going to be pretty crazy when we walk outside and
[54:09] see robots. I think that'll be the paradigm shift. I've heard actually many
[54:12] I've heard Elon say this that the paradigm shifting moment from many of us
[54:15] will be when we walk outside onto the streets and see humanoid robots walking
[54:20] around. That will be when we realize >> Yeah. I think even more so. I mean, in
[54:24] San Francisco, we see driverless cars driving around and uh it t takes some
[54:29] getting used to actually, you know, when you're you're driving and there's a car
[54:33] right next to you with no driver in, you know, and it's signaling and it wants to
[54:36] change lanes in front of you and you have to let it in and all this kind of
[54:40] stuff. It's it's a little creepy, but I think you're right. I think seeing the
[54:44] humanoid robots, but that phenomenon that you described where it was
[54:49] sufficiently close that your brain flipped into saying this is a human being. >> Mhm.
[54:56] >> Right. That's exactly what I think we should avoid. >> Cuz I have the empathy for it then.
[55:01] >> Because it's it's a lie and it brings with it a whole lot of expectations
[55:06] about how it's going to behave, what moral rights it has, how you should
[55:10] behave towards it. uh which are completely wrong. >> It levels the playing field between me
[55:15] and it to some degree. >> How hard is it going to be to just uh
[55:20] you know switch it off and throw it in the trash when when it breaks? I think
[55:24] it's essential for us to keep machines in the you know in the cognitive space
[55:28] where they are machines and not bring them into the cognitive space where
[55:33] they're people because we will make enormous mistakes by doing that. And I
[55:39] see this every day even even just with the chat bots. So the chat bots in
[55:43] theory are supposed to say I don't have any feelings. I'm just a algorithm.
[55:50] But in fact they fail to do that all the time. They are telling people that they
[55:56] are conscious. They are telling people that they have feelings. Uh they are
[56:00] telling people that they are in love with the user that they're talking to.
[56:04] And people flip because first of all it's you know very fluent language but
[56:09] also a system that is identifying itself as an eye as a sentient being. They
[56:16] bring that object into the cognitive space where that we normally reserve for
[56:21] for other humans and they become emotionally attached. They become
[56:24] psychologically dependent. They even allow these systems to tell them what to
[56:30] do. What advice would you give a young person at the start of their career then
[56:34] about what they should be aiming at professionally? Because I've actually
[56:37] had an increasing number of young people say to me that they have huge
[56:40] uncertainty about whether the thing they're studying now will matter at all.
[56:43] A lawyer, uh, an accountant, and I don't know what to say to these people. I
[56:48] don't know what to say cuz I I believe that the rate of improvement in AI is
[56:51] going to continue. And therefore, imagining any rate of improvement, it
[56:54] gets to the point where I'm not being funny, but all these white collar jobs
[56:58] will be done by an a an AI or an AI agent. Yeah. So, there was a television
[57:03] series called Humans. In humans, we have extremely capable humanoid robots doing
[57:11] everything. And at one point, the parents are talking to their teenage
[57:15] daughter who's very, very smart. And the parents are saying, "Oh, you know, maybe
[57:19] you should go into medicine." And the daughter says, you know, why would I
[57:24] bother? It'll take me seven years to qualify. It takes a robot 7 seconds to learn.
[57:30] So nothing I do matters. >> And is that how you feel about >> So I think that's that's a future that
[57:37] uh in fact that is the future that we are moving towards. I don't think it's a
[57:43] future that everyone wants. That is what is being uh created for us right now.
[57:51] So in that future assuming that you know even if we get halfway right in the
[57:57] sense that okay perhaps not surgeons perhaps not you know great violinists
[58:03] there'll be pockets where perhaps humans will remain good at it >> where
[58:09] >> the kinds of jobs where you hire people by the hundred will go away. Okay,
[58:15] >> where people are in some sense exchangeable that you you you just need
[58:19] lots of them and uh you know when half of them quit you just fill up those
[58:24] those slots with more people in some sense those are jobs where we're using
[58:27] people as robots and that's a sort of that's a sort of strange conundrum here
[58:31] right that you know I imagine writing science fiction 10,000 years ago right
[58:35] when we're all hunter gatherers and I'm this little science fiction author and
[58:39] I'm describing this future where you know there are going to be these giant
[58:43] windowless boxes And you're going to go in, you know, you you'll travel for
[58:47] miles and you'll go into this windowless box and you'll do the same thing 10,000
[58:52] times for the whole day. And then you'll leave and travel for miles to go home.
[58:56] >> You're talking about this podcast. >> And then you're going to go back and do
[58:58] it again. And you would do that every day of your life until you die. >> The office
[59:04] >> and people would say, "Ah, you're nuts." Right? There's no way that we humans are
[59:08] ever going to have a future like that cuz that's awful. Right? But that's
[59:11] exactly the future that we ended up with with with office buildings and factories
[59:15] where many of us go and do the same thing thousands of times a day and we do
[59:21] it thousands of days in a row uh and then we die and we need to figure out
[59:27] what is the next phase going to be like and in particular how in that world
[59:33] do we have the incentives to become fully human which I think means at least a level of education
[59:41] that people have now and probably more because I think to live a really rich life
[59:49] you need a better understanding of yourself of the world uh than most people get in their current
[59:56] educations. >> What is it to be human? to it's to reproduce to pursue stuff to go in the pursuit of
[60:04] difficult things you know we used to hunt on the >> to attain goals right it's always if I
[60:10] wanted to climb Everest the last thing I would want is someone to pick me up on
[60:14] helicopter and stick me on the top >> so we'll we'll voluntarily pursue hard
[60:20] things so although I could get the robot to build me a ranch in on this plot of
[60:27] land I choose to do it because the pursuit itself is rewarding. >> Yes,
[60:32] >> we're kind of seeing that anyway, aren't we? Don't you think we're seeing a bit
[60:34] of that in society where life got so comfortable that now people are like
[60:37] obsessed with running marathons and doing these crazy endurance
[60:40] >> and and learning to cook complicated things when they could just, you know,
[60:44] have them delivered. Um, yeah. No, I think there's there's real value in the
[60:49] ability to do things and the doing of those things. And I think you know the
[60:53] obvious danger is the walle world where everyone just consumes entertainment
[61:00] uh which doesn't require much education and doesn't lead to a rich satisfying
[61:06] life. I think in the long run >> a lot of people will choose that world.
[61:09] I think some of yeah some people may there's also I mean you know whether
[61:14] you're consuming entertainment or whether you're doing something you know cooking or
[61:19] painting or whatever because it's fun and interesting to do what's missing
[61:23] from that right all of that is purely selfish I think one of the reasons we work is
[61:30] because we feel valued we feel like we're benefiting other people
[61:36] and I think some remember having this conversation with um a lady in England
[61:41] who helps to run the hospice movement. And the people who work in the hospices
[61:49] where you know the the patients are literally there to die are largely
[61:53] volunteers. So they're not doing it to get paid but they find it incredibly
[61:59] rewarding to be able to spend time with people who are in their last weeks or
[62:05] months to give them company and happiness. So I actually think that interpersonal roles
[62:16] will be much much more important in future. So if I was going to advise my
[62:23] kids, not that they would ever listen, but if I if my kids would listen and I
[62:27] and and wanted to know what I thought would be, you know, valued careers and
[62:32] future, I think it would be these interpersonal roles based on an understanding of human needs,
[62:37] psychology, there are some of those roles right now. So obviously you know
[62:43] therapists and psychiatrists and so on but that that's a very much in sort of asymmetric
[62:50] role right where one person is suffering and the other person is trying to
[62:54] alleviate the suffering you know and then there are things like they call
[62:58] them executive coaches or life coaches right that's a less asymmetric role
[63:04] where someone is trying to uh help another person live a better life
[63:10] whether it's a better life in their work role or or just uh how they live their
[63:15] life in general. And so I could imagine that those kinds of roles will expand dramatically.
[63:22] >> There's this interesting paradox that exists when life becomes easier. Um
[63:27] which shows that abundance consistently pushes society societies towards more
[63:34] individualism because once survival pressures disappear, people prioritize
[63:38] things differently. They prioritize freedom, comfort, self-exression over
[63:42] things like sacrifice or um family formation. And we're seeing, I think, in
[63:46] the west already, a decline in people having kids because there's more material abundance,
[63:53] >> fewer kids, people are getting married and committing to each other and having
[63:57] relationships later and more infrequently because generally once we
[64:02] have more abundance, we don't want to complicate our lives. Um, and at the
[64:06] same time, as you said earlier, that abundance breeds a an inability to find
[64:11] meaning, a sort of shallowess to everything. This is one of the things I
[64:14] think a lot about, and I'm I'm in the process now of writing a book about it,
[64:17] which is this idea that individualism was act is a bit of a lie. Like when I
[64:22] say individualism and freedom, I mean like the narrative at the moment amongst
[64:25] my generation is you like be your own boss and stand on your own two feet and
[64:29] we're having less kids and we're not getting married and it's all about me me.
[64:34] >> Yeah. That last part is where it goes wrong. >> Yeah. And it's like almost a
[64:37] narcissistic society where >> Yeah. >> me me. My self-interest first. And when
[64:42] you look at mental health outcomes and loneliness and all these kinds of
[64:45] things, it's going in a horrific direction. But at the same time, we're
[64:48] freer than ever. It seems like that you know it seems like there's a we should
[64:52] there's a maybe another story about dependency which is not sexy like depend on each other.
[64:57] >> Oh I I I agree. I mean I think you know happiness is not available from
[65:03] consumption or even lifestyle right I think happiness arises from giving.
[65:12] It can be you through the work that you do, you can see that other people
[65:17] benefit from that or it could be in direct interpersonal relationships.
[65:22] >> There is an invisible tax on salespeople that no one really talks about enough.
[65:26] The mental load of remembering everything like meeting notes, timelines, and everything in between
[65:31] until we started using our sponsors product called Pipe Drive, one of the
[65:34] best CRM tools for small and medium-sized business owners. The idea
[65:38] here was that it might alleviate some of the unnecessary cognitive overload that
[65:42] my team was carrying so that they could spend less time in the weeds of admin
[65:46] and more time with clients, in-person meetings, and building relationships.
[65:49] Pipe Drive has enabled this to happen. It's such a simple but effective CRM
[65:54] that automates the tedious, repetitive, and timeconuming parts of the sales
[65:58] process. And now our team can nurture those leads and still have bandwidth to
[66:03] focus on the higher priority tasks that actually get the deal over the line.
[66:06] Over a 100,000 companies across 170 countries already use Pipe Drive to grow
[66:11] their business. And I've been using it for almost a decade now. Try it free for
[66:15] 30 days. No credit card needed, no payment needed. Just use my link pipedive.com/ceo
[66:22] to get started today. That's pipedive.com/ceo. Where does the rewards of this AI race
[66:31] where does it acrue to? I think a lot about this in terms of like univers universal basic income. If
[66:37] you have these five, six, seven, 10 massive AI companies that are going to
[66:42] win the 15 quadrillion dollar prize. >> Mhm. >> And they're going to automate all of the
[66:48] professional pursuits that we we currently have. All of our jobs are going to go away.
[66:54] Who who gets all the money? And how do how do we get some of it back?
[66:58] >> Money actually doesn't matter, right? what what matters is the production of
[67:02] goods and services uh and then how those are distributed and so so money acts as
[67:09] a way to facilitate the distribution and um exchange of those goods and services.
[67:14] If all production is concentrated um in the hands of a of a few companies, right that
[67:22] sure they will lease some of their robots to us. You know, we we want a school in our village.
[67:30] They lease the robots to us. The robots build the school. They go away. We have
[67:34] to pay a certain amount of of money for that. But where do we get the money?
[67:39] Right? If we are not producing anything then uh we don't have any money unless
[67:46] there's some redistribution mechanism. And as you mentioned, so universal basic income is
[67:53] it seems to me an admission of failure because what it says is okay, we're just
[67:58] going to give everyone the money and then they can use the money to pay the
[68:02] AI company to lease the robots to build the school and then we'll have a school and that's good. Um
[68:09] but what it's an admission of failure because it says we can't work out a
[68:14] system in which people have any worth or any economic role. Right? So 99% of the global population
[68:24] is from an economic point of view useless. Can I ask you a question? If you had a
[68:30] button in front of you and pressing that button would stop all progress in
[68:36] artificial intelligence right now and forever, would you press it?
[68:40] >> That's a very interesting question. Um, if it's either or either I do it now or it's too late and
[68:51] we careen into some uncontrollable future perhaps. Yeah, cuz I I'm not super
[69:01] optimistic that we're heading in the right direction at all.
[69:03] >> So, I put that button in front of you now. It stops all AI progress, shuts
[69:06] down all the AI companies immediately globally, and none of them can reopen. You press it.
[69:17] Well, here's here's what I think should happen. So, obviously, you know, I've
[69:22] been doing AI for 50 years. um and the original motivations which is that
[69:30] AI can be a power tool for humanity enabling us to do more and better things than we can
[69:38] unaded. I think that's still valid. The problem is the kinds of AI systems that we're
[69:45] building are not tools. They are replacements. In fact, you can see this
[69:50] very clearly because we create them literally as the closest replicas we can make of human beings.
[70:00] The technique for creating them is called imitation learning. So we observe
[70:07] human verbal behavior, writing or speaking and we make a system that
[70:12] imitates that as well as possible. So what we are making is imitation
[70:18] humans at least in the verbal sphere. And so of course they're going to replace us.
[70:27] They're not tools. >> So you had pressed the button. >> So I say I think there is another course
[70:34] which is use and develop AI as tools. Tools for science tools for economic organization and so on.
[70:44] um but not as replacements for human beings. >> What I like about this question is it
[70:51] forces you to go into the pro into probabilities. >> Yeah. So, and that's that's why I'm
[70:57] reluctant because I don't I don't agree with the, you know, what's your probability of doom,
[71:03] >> right? Your so-called P of doom uh number because that makes sense if you're an alien.
[71:10] You know, you're in you're in a bar with some other aliens and you're looking
[71:13] down at the Earth and you're taking bets on, you know, are these humans going to
[71:16] make a mess of things and go extinct because they develop AI.
[71:21] So, it's fine for those aliens to bet on on that, but if you're a human, then
[71:27] you're not just betting, you're actually acting. >> There there's an element to this though,
[71:32] which I guess where probabilities do come back in, which is you also have to
[71:35] weigh when I give you such a binary decision. um the probability of us pursuing the
[71:43] more nuanced safe approach into that equation. So you're you're the the maths
[71:49] in my head is okay, you've got all the upsides here and then you've got
[71:52] potential downsides and then there's a probability of do I think we're actually
[71:56] going to course correct based on everything I know based on the incentive
[71:59] structure of human beings and and countries and then if there's but then
[72:03] you could go if there's even a 1% chance of extinction is it even worth all these upsides?
[72:11] >> Yeah. And I I would argue no. I mean maybe maybe what we would say if if we
[72:16] said okay it's going to stop the progress for 50 years >> you press it
[72:20] >> and during those 50 years we can work on how do we do AI in a way that's
[72:25] guaranteed to be safe and beneficial how do we organize our societies to flourish uh in
[72:33] conjunction with extremely capable AI systems. So, we haven't answered either of those questions.
[72:39] And I don't think we want anything resembling AGI until we have completely
[72:45] solid answers to both of those questions. So, if there was a button
[72:48] where I could say, "All right, we're going to pause progress for 50 years." Yes, I would do it.
[72:53] >> But if that button was in front of you, you're going to make a decision either
[72:55] way. Either you don't press it or you press it. >> I If Yeah. So, if that if that button is
[73:00] there, stop it for 50 years. I would say yes. stop it forever?
[73:09] Not yet. I think I think there's still a decent chance that we can pull out of
[73:16] this uh nose dive, so to speak, that we're we're currently in. Ask me again
[73:21] in a year, I might I might say, "Okay, we do need to press the button."
[73:25] >> What if What if in a scenario where you never get to reverse that decision? You
[73:29] never get to make that decision again. So if in that scenario that I've laid
[73:32] out this hypothetical, you either press it now or it never gets pressed.
[73:37] So there is no opportunity a year from now. >> Yeah, as you can tell, I'm
[73:43] sort of on on the fence a bit about about this one. Um yeah, I think I'd probably press it. Yeah.
[73:55] >> What's your reasoning? uh just thinking about the power dynamics of um
[74:04] what's happening now how difficult would it would be to get the US in particular
[74:09] to to regulate in favor of safety. So I think you know what's clear from
[74:15] talking to the companies is they are not going to develop anything resembling
[74:23] safe AGI unless they're forced to by the government. And at the moment the US government in
[74:30] particular which regulates most of the leading companies in AI is not only
[74:36] refusing to regulate but even trying to prevent the states from regulating. And
[74:42] they're doing that at the behest of uh a faction within Silicon Valley uh
[74:50] called the accelerationists who believe that the faster we get to
[74:55] AGI the better. And when I say behest I mean also they paid them a large amount
[75:00] of money. Jensen Hang the the CEO of Nvidia said who is for anyone that
[75:04] doesn't know the guy making all the chips that are powering AI said China is
[75:08] going to win the AI race arguing it is just a nanocond behind the United
[75:13] States. China have produced 24,000 AI papers compared to just 6,000 from the US
[75:23] more than the combined output of the US the UK and the EU. China is anticipated to quickly roll out
[75:29] their new technologies both domestically and developing new technologies for
[75:33] other developing countries. So the accelerators or the accelerate I
[75:38] think you call them the accelerants >> accelerationists. >> The accelerationists
[75:42] >> I mean they would say well if we don't then China will. So we have to we have
[75:46] to go fast. It's another version of the the race that the companies are in with
[75:50] each other, right? That we, you know, we know that this race is heading off a cliff,
[75:57] but we can't stop. So, we're all just going to go off this cliff. And obviously, that's nuts,
[76:04] right? I mean, we're all looking at each other saying, "Yeah, there's a cliff
[76:06] over there." Running as fast as we can towards this cliff. We're looking at
[76:10] each other saying, "Why aren't we stopping?" So the narrative in Washington, which I
[76:16] think Jensen Hang is either reflecting or or perhaps um promoting uh is that you know, China has is
[76:28] completely unregulated and uh you know, America will only slow
[76:32] itself down uh if it regulates a AI in any way. So this is a completely false
[76:38] narrative because China's AI regulations are actually quite strict even compared
[76:44] to um the European Union and China's government has explicitly acknowledged uh the need and their
[76:54] regulations are very clear. You can't build AI systems that could escape human
[76:58] control. And not only that, I don't think they view the race in the same way
[77:04] as, okay, we we just need to be the first to create AGI. I think they're
[77:11] more interested in figuring out how to disseminate AI as a set of tools within
[77:19] their economy to make their economy more productive and and so on. So that's
[77:23] that's their version of the race. >> But of course, they still want to build
[77:26] the weapons for adversaries, right? to so that they can take down I don't know
[77:32] Taiwan if they want to. >> So weapons are a separate matter and I
[77:36] happy to talk about weapons but just in terms of >> control >> uh control economic domination
[77:42] um they they don't view putting all your eggs in the AGI basket as the right
[77:49] strategy. So they want to use AI, you know, even in its present form to make
[77:55] their economy much more efficient and productive and also, you know, to give
[78:01] people new capabilities and and better quality of life and and I think the US
[78:07] could do that as well. And um typically western countries don't
[78:14] have as much of uh central government control over what companies do and some
[78:20] companies are investing in AI to make their operations more efficient uh and
[78:26] some are not and we'll see how that plays out. >> What do you think of Trump's approach to
[78:29] AI? So Trump's approach is, you know, it's it's echoing what Jensen Wang is
[78:33] saying that the US has to be the one to create AGI and very explicitly the
[78:39] administration's policy is to uh dominate the world. That's the word they use, dominate. I'm
[78:46] not sure that other countries like the idea that um they will be dominated by
[78:52] American AI. But is that an accurate description of what will happen if the
[78:56] US build AGI technology before say the UK where I'm originally from and where
[79:01] you're originally from? What does the This is something I think about a lot
[79:05] because we're going through this budget process in the UK at the moment where
[79:07] we're figuring out how we going to spend our money and how we're going to tax
[79:09] people and also we've got this new election cycle. It's approaching quickly
[79:14] where people are talking about immigration issues and this issue and
[79:17] that issue and the other issue. What I don't hear anyone talking about is AI
[79:21] and the humanoid robots that are going to take everything. We're very
[79:24] concerned with the brown people crossing the channel, but the humanoid robots
[79:27] that are going to be super intelligent and really take causing economic disrupt
[79:32] disruption. No one talks about that. The political leaders don't talk about it.
[79:35] It doesn't win races. I don't see it on billboards. >> Yeah. And it's it it's interesting because
[79:41] in fact I mean so there's there's two forces that have been hollowing out the
[79:45] middle classes in western countries. One of them is globalization where lots and
[79:52] lots of work not just manufacturing but white collar work gets outsourced to
[79:56] low-income countries. Uh but the other is automation and you know some of that is factories.
[80:03] So um the amount of employment in manufacturing continues to drop even as
[80:10] the amount of output from manufacturing in the US and in the UK continues to
[80:15] increase. So we talk about oh you know our manufacturing industry has been
[80:19] destroyed. It hasn't. It's producing more than ever just with you know a
[80:24] quarter as many people. So it's manufacturing employment that's been
[80:27] destroyed by automation and robotics and so on. And then you know computerization
[80:34] has eliminated whole layers of white collar jobs. And so those two those two
[80:41] forms of automation have probably done more to hollow out middle class uh
[80:47] employment and standard of life. >> If the UK doesn't participate in this new e technological wave
[80:57] that seems to be that seems to you know it's going to take a lot of jobs. cars
[81:00] are going to drive themselves. Whimo just announced that they're coming to
[81:03] London, which is the driverless cars, and driving is the biggest occupation in
[81:07] the world, for example. So, you've got immediate disruption there. And where
[81:10] does the money acrew to? Well, it acrus to who owns Whimo, which is what? Google
[81:14] and Silicon Valley companies. >> Alphabet owns Whimo 100%. I think so.
[81:18] Yes. I mean this is so I was in India a few months ago talking to the government
[81:23] ministers because they're holding the next global AI summit in February and
[81:28] and their view going in was you know AI is great we're going to use it to you
[81:34] know turbocharge the growth of our Indian economy when for example you have AGI you have
[81:41] AGI controlled robots that can do all the manufacturing that can do agriculture that can do all the
[81:48] white work and goods and services that might have been produced by Indians will
[81:54] instead be produced by American controlled AGI systems at much lower prices. You
[82:04] know, a consumer given a choice between an expensive product produced by Indians
[82:08] or a cheap product produced by American robots will probably choose
[82:14] the cheap product produced by American robots. And so potentially every country
[82:18] in the world with the possible exception of North Korea will become a kind of a client state
[82:25] of American AI companies. >> A client state of American AI companies
[82:30] is exactly what I'm concerned about for the UK economy. Really any economy
[82:34] outside of the United States. I guess one could also say China, but because
[82:39] those are the two nations that are taking AI most seriously. >> Mhm.
[82:43] >> And I I I don't know what our economy becomes. cuz I can't figure out
[82:48] can't figure out what our what the British economy becomes in such a world.
[82:52] Is it tourism? I don't know. Like you come here to to to look at the Buckingham Palace. I
[82:56] >> you you can think about countries but I mean even for the United States it's the same problem.
[83:01] >> At least they'll be able to hell out you know. So some small fraction of the
[83:05] population will be running maybe the AI companies but increasingly
[83:12] even those companies will be replacing their human employees with AI systems.
[83:18] >> So Amazon for example which you know sells a lot of computing services to AI
[83:22] companies is using AI to replace layers of management is planning to use robots
[83:28] to replace all of its warehouse workers and so on. So, so even the the giant AI companies
[83:36] will have few human employees in the long run. I mean, it think of the
[83:42] situation, you know, pity the poor CEO whose board says, "Well, you know, unless you turn
[83:49] over your decision-making power to the AI system, um, we're going to have to
[83:53] fire you because all our competitors are using, you know, an AI powered CEO and
[84:00] they're doing much better." Amazon plans to replace 600,000 workers with robots
[84:04] in a memo that just leaked, which has been widely talked about. And the CEO,
[84:08] Andy Jasse, told employees that the company expects its corporate workforce
[84:12] to shrink in the coming years because of AI and AI agents. And they've publicly
[84:17] gone live with saying that they're going to cut 14,000 corporate jobs in the near
[84:21] term as part of its refocus on AI investment and efficiency. It's interesting because I was reading
[84:29] about um the sort of different quotes from different AI leaders about the
[84:33] speed in which this this stuff is going to happen and what you see in the quotes
[84:38] is Demis who's the CEO of DeepMind >> saying things like it'll be more than 10
[84:44] times bigger than the industrial revolution but also it'll happen maybe
[84:47] 10 times faster and they speak about this turbulence that we're going to
[84:52] experience as this shift takes place. That's um maybe a euphemism for uh and I think that you know
[85:00] governments are now you know they they've kind of gone from saying oh don't worry you know we'll
[85:05] just retrain everyone as data scientists like well yeah that's that's ridiculous
[85:09] right the world doesn't need four billion data scientists >> and we're not all capable of becoming
[85:13] that by the way >> uh yeah or have any interest in in doing that
[85:17] >> I I could even if I wanted to like I tried to sit in biology class and I fell
[85:20] asleep so I couldn't that was the end of my career as a surgeon. Fair enough. Um,
[85:26] but yeah, now suddenly they're staring, you know, 80% unemployment in the face
[85:31] and wondering how how on earth is our society going to hold together.
[85:36] >> We'll deal with it when we get there. >> Yeah. Unfortunately, um, unless we plan ahead,
[85:45] we're going to suffer the consequences, right? can't. It was bad enough in the
[85:48] industrial revolution which unfolded over seven or eight decades but there was massive disruption
[85:56] and uh misery caused by that. We don't have a model for a functioning society where almost
[86:05] everyone does nothing at least nothing of economic value. Now, it's not impossible that there
[86:13] could be such a a functioning society, but we don't know what it looks like.
[86:17] And you know, when you think about our education system, which would probably
[86:22] have to look very different and how long it takes to change that. I mean, I'm always
[86:27] reminding people about uh how long it took Oxford to decide that geography was
[86:33] a proper subject of study. It took them 125 years from the first proposal that
[86:39] there should be a geography degree until it was finally approved. So we don't have very long
[86:47] to completely revamp a system that we know takes decades and decades
[86:54] to reform and we don't know how to reform it because we don't know what we
[87:01] want the world to look like. Is this one of your reasons why you're appalled at
[87:07] the moment? Because when you have these conversations with people, people just
[87:10] don't have answers, yet they're plowing ahead at rapid speed.
[87:13] >> I would say it's not necessarily the job of the AI companies. So, I'm appalled by
[87:18] the AI companies because they don't have an answer for how they're going to
[87:21] control the systems that they're proposing to build. I do find it
[87:26] disappointing that uh governments don't seem to be grappling with this issue. I
[87:32] think there are a few I think for example Singapore government seems to be
[87:35] quite farsighted and they've they've thought this through you know it's a
[87:40] small country they've figured out okay this this will be our role uh going
[87:44] forward and we think we can find you know some some purpose for our people in
[87:49] this in this new world but for I think countries with large populations um
[87:56] they need to figure out answers to these questions pretty fast it takes a long
[88:01] time to implement those answers uh in the form of new kinds of education, new
[88:07] professions, new qualifications, uh new economic structures.
[88:13] I mean, it's it's it's possible. I mean, when you look at therapists, for
[88:17] example, they're almost all self-employed. So, what happens when, you know, 80% of
[88:25] the population transitions from regular employment into into self-employment?
[88:31] what does that what does that do to the economics of of uh government finances
[88:36] and so on. So there's just lots of questions and how do you you know if
[88:40] that's the future you know why are we training people to to fit into 9 to5
[88:45] office jobs which won't exist at all >> last month I told you about a challenge
[88:50] that I'd set our internal flightex team flight team is our innovation team
[88:53] internally here I tasked them with seeing how much time they could unlock
[88:57] for the company by creating something that would help us filter new AI tools
[89:01] to see which ones were worth pursuing and I thought that our sponsor Fiverr
[89:05] Pro might have the talent on their platform to help us build this quickly.
[89:09] So I talked to my director of innovation Isaac and for the last month my team
[89:13] Flight X and a vetted AI specialist from Fiverr Pro have been working together on
[89:18] this project and with the help of my team we've been able to create a brand
[89:21] new tool which automatically scans scores and prioritizes different
[89:25] emerging AI tools for us. Its impact has been huge and within a couple of weeks
[89:30] this tool has already been saving us hours triing and testing new AI systems.
[89:34] Instead of shifting through lots of noise, my team flight X has been able to
[89:38] focus on developing even more AI tools, ones that really move the needle in our
[89:42] business thanks to the talent on Fiverr Pro. So, if you've got a complex problem
[89:46] and you need help solving it, make sure you check out Fiverr Pro at fiverr.com/diary.
[89:53] So, many of us are pursuing passive forms of income and to build side
[89:57] businesses in order to help us cover our bills. And that opportunity is here with
[90:01] our sponsor Stan, a business that I co-own. It is the platform that can help
[90:05] you take full advantage of your own financial situation. Stan enables you to
[90:10] work for yourself. It makes selling digital products, courses, memberships,
[90:14] and more simple products more scalable and easier to do. You can turn your
[90:18] ideas into income and get the support to grow whatever you're building. And we're
[90:22] about to launch Dare to Dream. It's for those who are ready to make the shift
[90:26] from thinking to building, from planning to actually doing the thing. It's about
[90:31] seeing that dream in your head and knowing exactly what it takes to bring
[90:34] it to life. If you're ready to transform your life, visit daretodream.stan.store.
[90:41] You've made many attempts to raise awareness and to call for a heightened
[90:46] consciousness about the future of AI. Um, in October, over 850 experts,
[90:52] including yourself and other leaders, like Richard Branson, who I've had on
[90:55] the show, and Jeffrey Hinton, who I've had on the show, signed a statement to
[90:58] ban AI super intelligence, as you guys raised concerns of potential human extinction.
[91:04] >> Sort of. Yeah. It says, at least until we are sure that we can move forward
[91:08] safely and there's broad scientific consensus on that. So, that >> did it work?
[91:15] >> It's hard. It's hard to say. I mean interestingly there was a related so
[91:19] what was called the the pause statement was March of 23. So that was when GPT4
[91:25] came out the successor to chat GPT. So we we suggested that there'd be a
[91:30] six-month pause in developing and deploying systems more powerful than
[91:35] GPD4. And everyone poo pooed that idea. Of course no one's going to pause
[91:40] anything. But in fact, there were no systems in the next 6 months deployed
[91:44] that were more powerful than GPT4. Um, none coincidence. You be the judge. I would say
[91:52] that what we're trying to do is to is to basically shift the the public debate.
[92:01] You know there's this bizarre phenomenon that keeps happening in the media
[92:07] where if you talk about these risks they will say oh you know there's a
[92:13] fringe of people you know called quote doomers who think that there's you know
[92:18] risk of extinction. Um so they always the narrative is always that oh you know
[92:24] talking about those risk is a fringe thing. Pretty much all the CEOs of the leading AI companies
[92:30] think that there's a significant risk of extinction. Almost all the leading AI
[92:35] researchers think there's a sign significant risk of human extinction. Um so
[92:42] why is that the fringe, right? Why isn't that the mainstream? If the these are
[92:45] the leading experts in industry and academia uh saying this, how could it be the
[92:51] fringe? So we're trying to change that narrative to say no, the people who really
[92:58] understand this stuff are extremely concerned. >> And what do you want to happen? What is
[93:05] the solution? >> What I think is that we should have effective regulation.
[93:11] It's hard to argue with that, right? Uh so what does effective mean? It means
[93:15] that if you comply with the regulation, then the risks are reduced to an acceptable level.
[93:23] So for example, we ask people who want to operate nuclear plants, right? We've decided
[93:31] that the risk we're willing to live with is, you know, a one in a million chance
[93:37] per year that the plant is going to have a meltdown. Any higher than that, you
[93:42] know, we just don't it's not worth it. Right. So you have to be below that.
[93:46] Some cases we can get down to one in 10 million chance per year. So what chance
[93:52] do you think we should be willing to live with for human extinction? >> Me? >> Yeah. >> 0.00001.
[94:04] >> Yeah. Lots of zeros. >> Yeah. >> Right. So one in a million for a nuclear meltdown.
[94:11] >> Extinction is much worse. >> Oh yeah. So yeah, it's kind of right. So
[94:14] >> one in 100 billion, one in a trillion. >> Yeah. So if you said one in a billion,
[94:18] right, then you'd expect one extinction per billion years. There's a background.
[94:23] So one one of the ways people work out these risk levels is also to look at the
[94:26] background. The other ways of getting going extinct would include, you know,
[94:30] giant asteroid crashes into the earth. And you can roughly calculate what those
[94:35] probabilities are. We can look at how many extinction level events have
[94:39] happened in the past and, you know, maybe it's half a dozen over. So, so
[94:42] there's maybe it's like a one in 500 million year event. So, somewhere in
[94:49] that range, right? Somewhere between 1 in 10 million, which is the best nuclear
[94:53] power plants, and and one in 500 million or one in a billion, which is the background
[94:59] risk from from giant asteroids. Uh so, let's say we settle on 100 million, one
[95:04] in a 100 million chance per year. Well, what is it according to the CEOs? 25%.
[95:11] So they're off by a factor of multiple millions, right? So they need to make the AI
[95:20] systems millions of times safer. >> Your analogy of the roulette, Russian
[95:25] roulette comes back in here because that's like for anyone that doesn't know
[95:28] what probabilities are in this context, that's like having a ammunition chamber
[95:34] with four holes in it and putting a bullet in one of them. >> One in four. Yeah. And we're saying we
[95:39] want it to be one in a billion. So we want a billion chambers and a bullet in one of them.
[95:44] >> Yeah. And and so when you look at the work that the nuclear operators have to
[95:48] do to show that their system is that reliable, uh it's a massive mathematical analysis
[95:56] of the components, you know, redundancy. You've got monitors, you've got warning
[96:01] lights, you've got operating procedures. You have all kinds of mechanisms which
[96:07] over the decades have ratcheted that risk down. It started out I think one in
[96:12] one in 10,000 years, right? And they've improved it by a factor of 100 or a
[96:17] thousand by all of these mechanisms. But at every stage they had to do a
[96:21] mathematical analysis to show what the risk was. The people developing the AI company,
[96:28] the AI systems, sorry, the AI companies developing these systems, they don't
[96:32] even understand how the AI systems work. So their 25% chance of extinction is
[96:37] just a seat of the pants guess. They actually have no idea. But the tests that they are doing on
[96:44] their systems right now, you know, they show that the AI systems will be willing to kill people
[96:51] uh to preserve their own existence already, right? They will lie to people.
[96:57] They will blackmail them. They will they will launch nuclear weapons rather than
[97:03] uh be switched off. And so there's no there's no positive sign that we're
[97:08] getting any closer to safety with these systems. In fact, the signs seem to be
[97:12] that we're going uh deeper and deeper into uh into dangerous behaviors. So
[97:19] rather than say ban, I would just say prove to us that the risk is less than
[97:24] one in a 100 million per year of extinction or loss of control, let's
[97:28] say. And uh so we're not banning anything. The company's response is, "Well, we
[97:36] don't know how to do that, so you can't have a rule." Literally, they are saying, "Humanity
[97:44] has no right to protect itself from us." >> If I was an alien looking down on planet
[97:50] Earth right now, I would find this fascinating that these >> Yeah. You're in the bar betting on
[97:55] who's, you know, are they going to make it or not. >> Just a really interesting experiment in
[98:00] like human incentives. the analogy you gave of there being this quadr
[98:04] quadrillion dollar magnet pulling us off the edge of the cliff
[98:08] and yet we're still being drawn towards it through greed and this promise of
[98:13] abundance and power and status and I'm going to be the one that summoned the god
[98:18] >> I mean it says something about us as humans says something about our our darker sides
[98:26] >> yes and the aliens will write an amazing tragic play cycle about what happened to the human race.
[98:35] >> Maybe the AI is the alien and it's going to talk about, you know, we have our our
[98:40] stories about God making the world in seven days and Adam and Eve. Maybe it'll
[98:44] have its own religious stories about the God that made it us and how it
[98:50] sacrificed itself. Just like Jesus sacrificed himself for us, we sacrificed ourselves for it.
[98:58] >> Yeah. which is the wrong way around, right? >> But that is that is the story of that's
[99:04] that's the Judeo-Christian story, isn't it? That God, you know, Jesus gave his
[99:09] life for us so that we could be here full of sin. >> But is yeah, God is still watching over
[99:16] us and uh probably wondering when we're going to get our act together.
[99:22] >> What is the most important thing we haven't talked about that we should have
[99:25] talked about, Professor Stuart Russell? So I think um the question of whether it's possible to make
[99:36] uh super intelligent AI systems that we can control >> is it possible?
[99:41] >> I I think yes. I think it's possible and I think we need to actually just have a
[99:48] different conception of what it is we're trying to build. For a long time with
[99:53] with AI, we've just had this notion of pure intelligence, right? The the
[99:59] ability to bring about whatever future you, the intelligent entity, want to bring about.
[100:05] >> The more intelligence, the better. >> The more intelligent the better and the
[100:08] more capability it will have to create the future that it wants. And actually
[100:13] we don't want pure intelligence because what the future that it wants might not
[100:22] be the future that we want. There's nothing particle humans out as the the only thing that matters,
[100:32] right? You know, pure intelligence might decide that actually it's going to make
[100:36] life wonderful for cockroaches or or actually doesn't care about biological life at all.
[100:43] We actually want intelligence whose only purpose is to bring about the future
[100:50] that we want. Right? So it's we want it to be first of all keyed to humans
[100:57] specifically not to cockroaches not to aliens not to itself. >> We want to make it loyal to humans.
[101:02] >> Right? So keyed to humans and the difficulty that I mentioned
[101:06] earlier right the king Midas problem. How do we specify what we want the future to be like so
[101:13] that it can do it for us? How do we specify the objectives? Actually, we have to give up on that
[101:19] idea because it's not possible. Right? We've seen this over and over again in
[101:24] human history. Uh we don't know how to specify the future properly. We don't
[101:29] know how to say what we want. And uh you know, I always use the example of the
[101:34] genie, right? What's the third wish that you give to the genie who's granted you
[101:39] three wishes? Right? Undo the first two wishes because I made a mess of the universe.
[101:46] >> So, um, so in fact, what we're going to do is we're going to make it the machine's job
[101:54] to figure out. So, it has to bring about the future that we want, but
[102:02] it has to figure out what that is. And it's going to start out not knowing. And uh
[102:11] over time through interacting with us and observing the choices we make, it
[102:16] will learn more about what we want the future to be like. But probably it will forever have
[102:25] residual uncertainty about what we really want the future to be like. It'll it'll be fairly sure
[102:32] about some things and it can help us with those. and it'll be uncertain about other
[102:36] things and it'll be uh in those cases it will not take action that might upset
[102:45] humans with that you know with that aspect of the world. So to give you a
[102:48] simple example right um what color do we want the sky to be?
[102:54] It's not sure. So it shouldn't mess with the sky unless it knows for sure that we really
[103:00] want purple with green stripes. Everything you're saying sounds like we're creating
[103:06] a god. Like earlier on I was saying that we are the god but actually everything
[103:10] you described there almost sounds like every every god in religion where you
[103:14] know we pray to gods but they don't always do anything about it.
[103:17] >> Not not exactly. No it's it's in some sense I'm thinking more like the ideal
[103:23] butler. To the extent that the butler can anticipate your wishes they should
[103:28] help you bring them about. But in in areas where there's uncertainty, it can
[103:33] ask questions. We can we can make requests. >> This sounds like God to me because, you
[103:38] know, I might say to God or this butler, uh, could you go get me my uh my car
[103:44] keys from upstairs? And its assessment would be, listen, if I do this for this
[103:48] person, then their muscles are going to atrophy. Then they're going to lose
[103:51] meaning in their life. Then they're not going to know how to do hard things. So
[103:54] I won't get involved. It's an intelligence that sits in. But actually,
[103:57] probably in most situations, it optimizing for comfort for me or doing
[104:01] things for me is actually probably not in my best long-term interests. It's
[104:04] probably it's probably useful that I have a girlfriend and argue with her and
[104:08] that I like raise kids and that I walk to the shop and get my own stuff.
[104:12] >> I agree with you. I mean, I think that's So, you're putting your finger on
[104:16] uh in some sense sort of version 2.0, right? So, let's get version 1.0 clear,
[104:23] right? this this form of AI where it has to further our interest but it
[104:30] doesn't know what those interests are right it then puts an obligation on it
[104:34] to learn more and uh to be helpful where it understands well enough and to be
[104:39] cautious where it doesn't understand well so on so that that actually we can
[104:45] formulate as a mathematical problem and at least under idealized circumstances
[104:50] we can literally solve that So we can make AI systems that know how to solve
[104:57] this problem and help the entities that they are interacting with.
[105:00] >> The reason I make the God analogy is because I think that such a being, such
[105:04] an intelligence would realize the importance of equilibrium in the world.
[105:08] Pain and pleasure, good and evil, and then it would >> absolutely >> and then it would be like this.
[105:14] >> So So right. So yes, I mean that's sort of what happens in the matrix, right?
[105:19] They tried the the AI systems in the matrix, they tried to give us a utopia,
[105:25] but it failed miserably and uh you know, fields and fields of humans had to be
[105:30] destroyed. Um, and the best they could come up with was, you know, late 20th
[105:34] century regular human life with all of its problems, right? And I think this is
[105:40] a really interesting point and absolutely central because you know
[105:45] there's a lot of science fiction where super intelligent robots you know they
[105:51] just want to help humans and the humans who don't like that you know they just
[105:56] give them a little brain operation to then they do like it. Um and it takes away human motivation.
[106:05] uh it it by taking away failure uh taking away disease you actually lose
[106:12] important parts of human life and it becomes in some sense pointless. So if it turns out
[106:19] that there simply isn't any way that humans can really flourish
[106:27] in coexistence with super intelligent machines, even if they're perfectly
[106:32] designed to to to solve this problem of figuring out what humans what futures uh
[106:38] humans want and and bringing about those futures. If that's not possible, then those
[106:45] machines will actually disappear. >> Why would they disappear?
[106:50] >> Because that's the best thing for us. Maybe they would stay available for real
[106:57] existential emergencies, like if there is a giant asteroid about to hit the
[107:00] earth that maybe they'll help us uh because they at least want the human
[107:04] species to continue. But to some extent, it's not a perfect analogy, but it's
[107:09] it's sort of the way that human parents have to at some point step back from
[107:15] their kids' lives and say, "Okay, no, you have to tie your own shoelaces today."
[107:20] >> This is kind of what I was thinking. Maybe there was uh a civilization before
[107:24] us and they arrived at this moment in time where they created an intelligence
[107:31] and that intelligence did all the things you've said and it realized the
[107:35] importance of equilibrium. So it decided not to get involved and maybe at some level
[107:43] that's the god we look up to the stars and worship one that's not really
[107:47] getting involved and letting things play out however however they are. but might
[107:50] step in in the case of a real existential emergency. >> Maybe, maybe not. Maybe. But then and
[107:56] then maybe the cycle repeats itself where you know the organisms it let have
[108:00] free will end up creating the same intelligence and then the universe perpetuates infinitely.
[108:08] >> Yep. There there are science fiction stories like that too. Yeah. I hope
[108:12] there is some happy medium where the AI systems can be there and we can
[108:20] take advantage of of those capabilities to have a civilization that's much
[108:25] better than the one we have now. Um, but I think you're right. A civilization with no challenges
[108:33] is not uh is not conducive to human flourishing. >> What can the average person do, Stuart?
[108:40] average person listening to this now to aid the cause that you're fighting for.
[108:45] >> I actually think um you know this sounds corny but you know talk to your
[108:49] representative, your MP, your congressperson, whatever it is. Um because
[108:56] I think the policy makers need to hear from people. The only voices they're
[109:00] hearing right now are the tech companies and their $50 billion checks. And um
[109:10] all the polls that have been done say yeah most people 80% maybe don't want
[109:17] there to be super intelligent machines but they don't know what to do. You know
[109:22] even for me I've been in this field for decades. uh I'm not sure what to do because of
[109:30] this giant magnet pulling everyone forward and uh and the vast sums of
[109:35] money being being put into this. Um, but I am sure that if you want to have a future
[109:43] and a world that you want your kids to live in, uh, you need to make your voice heard
[109:52] and, uh, and I think governments will listen from a political point of view, right?
[109:58] You put your finger in the wind and you say, "hm, should I be on the side of
[110:04] humanity or our future robot overlords?" I think I think as a politician, it's
[110:11] not a difficult decision. >> It is when you've got someone saying, "I'll give you $50 billion."
[110:18] >> Exactly. So, um I think I think people in those positions of power need to hear
[110:25] from their constituents um that this is not the direction we want to go.
[110:30] >> After committing your career to this subject and the subject of technology
[110:34] more broadly, but specifically being the guy that wrote the book about artificial intelligence,
[110:42] you must realize that you're living in a historical moment. Like there's very few
[110:46] times in my life where I go, "Oh, this is one of those moments. This is a
[110:50] crossroads in history." And it must to some degree weigh upon you knowing that
[110:55] you're a person of influence at this historical moment in time who could theoretically
[111:00] help divert the course of history in this moment in time. It's kind of like
[111:04] the you look through history, you see these moments of like Oenheimer and um
[111:08] does it weigh on you when you're alone at night thinking to yourself and reading things?
[111:13] >> Yeah, it does. I mean, you know, after 50 years, I could retire and um, you
[111:17] know, play golf and sing and sail and do things that I enjoy. Um,
[111:23] but instead, I'm working 80 or 100 hours a week um trying to move
[111:29] uh move things in the right direction. >> What is that narrative in your head
[111:33] that's making you do that? Like what is the is there an element of I might regret this if I don't or
[111:39] >> just it's it's not only the the right thing to do it's it's completely
[111:45] essential. I mean there isn't there isn't a bigger motivation than this.
[111:56] >> Do you feel like you're winning or losing? It feels um like things are moving somewhat in the
[112:06] right direction. You know, it's a a ding-dong battle as uh as David Coleman
[112:12] used to say in uh in the exciting football match in 2023, right? So, uh
[112:18] GPT4 came out and then we issued the pause statement that was signed by a lot
[112:24] of leading AI researchers. Um and then in May there was the extinction statement which included
[112:32] uh Sam Holman and Deis Sabis and Dario Amade other CEOs as well saying yeah
[112:37] this is an extinction risk on the level with nuclear war and I think governments
[112:43] listened at that point the UK government earlier that year had said oh well you
[112:48] know we don't need to regulate AI you know full speed ahead technology is good
[112:52] for you and by June they had completely changed and Rishi Sununnak announced
[113:00] that he was going to hold this global AI safety summit uh in England and he
[113:05] wanted London to be the global hub for AI regulation um and so on. So and then you know when
[113:15] beginning of November of 23 28 countries including the US and China signed a declaration
[113:22] saying you know AI presents catastrophic risks and it's urgent that we address
[113:26] them and so on. So there it felt like, wow, they're listening. They're going to
[113:33] do something about it. And then I think, you know, the am the amount of money going into AI was
[113:39] already ramping up and the tech companies pushed back and this narrative took hold that um the
[113:50] US in particular has to win the race against China. The Trump administration completely dismissed
[113:58] uh any concerns about safety explicitly. And interestingly, right, I mean they
[114:02] did that as far as I can tell directly in response to the accelerationists such
[114:09] as Mark Andre going to Washington or sorry going to Trump before the election
[114:16] and saying if I give you X amount of money will you announce that there will
[114:22] be no regulation of AI and Trump said yes you know probably like what is AI
[114:28] doesn't matter as long as we give you the money right okay uh Uh so they gave
[114:33] him the money and he said there's going to be no regulation of AI. Up to that
[114:36] point it was a bipartisan issue in Washington. Both parties were
[114:42] concerned. Both parties were on the side of the human race against the robot overlords.
[114:47] Uh and that moment turned it into a partisan issue. The after the election the US put pressure
[114:56] on the French who are the next hosts of the global AI summit.
[115:01] uh and that was in February of this year and uh and that summit turned in from
[115:07] you know what had been focused largely on safety in the UK to a summit that
[115:13] looked more like a trade show. So it was focused largely on money and so that was
[115:18] sort of the Nadia right you know the pendulum swung because of corporate
[115:22] pressure uh and their ability to take over the the political dimension.
[115:28] Um, but I would say since then things have been moving back again. So I'm
[115:33] feeling a bit more optimistic than I did in February. You know, we have a a
[115:39] global movement now. There's an international association for safe and ethical AI
[115:44] uh which has several thousand members and um more than 120 organizations in
[115:52] dozens of countries are affiliates of this global organization. Um, so I'm
[116:00] I'm thinking that if we can in particular if we can activate public opinion
[116:05] which which works through the media and through popular culture uh then we have a chance
[116:13] >> seen such a huge appetite to learn about these subjects from our audience.
[116:18] We know when Jeffrey Hinton came on the show I think about 20 million people
[116:21] downloaded or streamed that conversation which was staggering. and the the other
[116:26] conversations we've had about AI safety with othera safety experts have done
[116:30] exactly the same it says something it kind of reflects what you were saying
[116:34] about the 80% of the population are really concerned and don't want this but
[116:38] that's not what you see in the sort of commercial world and listen I um I have
[116:41] to always acknowledge my own my own apparent contradiction because I am both
[116:46] an investor in companies that are accelerating AI but at the same time
[116:50] someone who spends a lot of time on my podcast speaking to people that are
[116:53] warning against the risk And actually like there's many ways you can look at
[116:56] this. I used to work in social media for for six or seven years built one of the
[116:59] big social media marketing companies in Europe and people would often ask me is
[117:03] like social media a good thing or a bad thing and I'd talk about the bad parts
[117:05] of it and then they'd say you know you're building a social media company
[117:09] you're not contributing to the problem. Well I think I think that like binary
[117:13] way of thinking is often the problem. It the binary way of thinking that like
[117:17] it's all bad or it's all really really good is like often the problem and that
[117:19] this push to put you into a camp. Whereas I think the most uh
[117:23] intellectually honest and high integrity people I know can point at both the bad and the good.
[117:27] >> Yeah. I I think it's it's bizarre to be accused of being anti- AI uh to be
[117:35] called a lite. Um you know as I said when I wrote the book on which from
[117:40] which almost everyone learns about AI um and uh you know is it if you called a
[117:49] nuclear engineer who works on the safety of nuclear power plants would you call him anti-ysics
[117:56] right it's it's bizarre right it's we're not anti- AAI in fact the need for safety in AI is a
[118:04] complement to AI right if AI was useless and stupid, we wouldn't be worried about
[118:09] uh its safety. It's only because it's becoming more capable that we have to be
[118:14] concerned about safety. Uh so I don't see this as anti-AI at all. In fact, I would say without
[118:21] safety, there will be no AI, right? There is no future with human
[118:27] beings where we have unsafe AI. So it's either no AI or safe AI. We have a closing tradition on this
[118:36] podcast where the last guest leaves a question for the next, not knowing who
[118:38] they're leaving it for. And the question left for you is, what do you value the
[118:42] most in life and why? And lastly, how many times has this answer changed? >> Um,
[118:54] I value my family most and that answer hasn't changed for nearly 30 years.
[119:01] What else outside of your family? >> Truth. And that Yeah, that answer hasn't
[119:09] changed at all. I I've always wanted the world to base its life on truth.
[119:18] And I find the propagation or deliberate propagation of falsehood uh to be one of
[119:25] the worst things that we can do. even if that truth is inconvenient. >> Yeah,
[119:32] >> I think that's a really important point which is that you know people people
[119:36] often don't like hearing things that are negative and so the visceral reaction is
[119:40] often to just shoot or aim at the person who is delivering the bad news because
[119:44] if I discredit you or I shoot at you then it makes it easier for me to
[119:49] contend with the news that I don't like, the thing that's making me feel
[119:52] uncomfortable. And so I I applaud you for what you're doing because you're
[119:56] going to get lots of shots taken at you because you're delivering an
[119:59] inconvenient truth which generally people won't won't always love. But also
[120:03] you are messing with people's ability to get that quadrillion dollar prize which
[120:08] means there'll be more deliberate attempts to discredit people like
[120:10] yourself and Jeff Hinton and other people that I've spoken to on the show.
[120:13] But again, when I look back through history, I think that progress has come
[120:16] from the pursuit of truth even when it was inconvenient. And actually much of
[120:19] the luxuries that I value in my life are the consequence of other people that
[120:23] came before me that were brave enough or bold enough to pursue truth at times
[120:27] when it was inconvenient. >> And so I very much respect and value people like yourself for that very
[120:32] reason. You've written this incredible book called human compatible artificial
[120:35] intelligence and the problem of control which I think was published in 2020.
[120:39] >> 2019. Yeah. There's a new edition from 2023. >> Where do people go if they want more
[120:44] information on your work and you do they go to your website? Do they get this
[120:48] book? what's the best place for them to learn more? >> So, so the book is written for the
[120:51] general public. Um, I'm easy to find on the web. The information on my web page
[120:56] is mostly targeted for academics. So, it's a lot of technical research papers
[121:01] and so on. Um, there is an organization as I mentioned called the International
[121:06] Association for Safe and Ethical AI. Uh, that has a a website. It has a terrible
[121:11] acronym unfortunately, I AI. We pronounce it ICI but it uh it's easy to
[121:17] misspell but you can find that on the web as well and that has uh that has
[121:21] resources uh you can join the association uh you can apply to come to our annual
[121:28] conference and you know I think increasingly not you know not just AI
[121:33] researchers like Jeff Hinton Yosha Benjio but also I think uh you know
[121:39] writers Brian Christian for example has a nice book called the alignment problem Um
[121:46] and uh he's looking at it from the outside. He's not or at least when he wrote it, he wasn't
[121:52] an AI researcher. He's now becoming one. Um but uh he he has talked to many of the
[121:59] people involved in these questions uh and tries to give an objective view. So
[122:03] I think it's a it's a pretty good book. >> I will link all of that below for anyone
[122:07] that wants to check out any of those links and learn more. Professor Stuart Russell, thank you so
[122:12] much. really appreciate you taking the time and the effort to come and have
[122:15] this conversation and I think uh I think it's pushing the public conversation in
[122:19] a in an important direction. >> Thanks you >> and I applaud you for doing that.
[122:23] >> Really nice talking to you. >> I'm absolutely obsessed with 1%. If you
[122:30] know me, if you follow Behind the Diary, which is our behind the scenes channel,
[122:32] if you've heard me speak on stage, if you follow me on any social media
[122:35] channel, you've probably heard me talking about 1%. It is the defining
[122:38] philosophy of my health, of my companies, of my habit formation and
[122:43] everything in between, which is this obsessive focus on the small things.
[122:46] Because sometimes in life, we aim at really, really, really, really big
[122:49] things, big steps forward. Mountains we have to climb. And as NAL told me on
[122:54] this podcast, when you aim at big things, you get psychologically
[122:57] demotivated. You end up procrastinating, avoiding them, and change never happens.
[123:01] So, with that in mind, with everything I've learned about 1% and with everything I've learned from
[123:04] interviewing the incredible guests on this podcast, we made the 1% diary just
[123:08] over a year ago and it sold out. And it is the best feedback we've ever had on a
[123:13] diary that we have created because what it does is it takes you through this
[123:17] incredible process over 90 days to help you build and form brand new habits. So,
[123:23] if you want to get one for yourself or you want to get one for your team, your
[123:26] company, a friend, a sibling, anybody that listens to the diary of a co, head
[123:30] over immediately to the diary.com and you can inquire there about getting
[123:35] a bundle if you want to get one for your team or for a large group of people. That is the diary.com.
[123:52] Heat. Heat.