[0:00] concept of AGI is a bunch of baloney. [0:02] Humans are this magical species with all [0:05] of these skills and capabilities that we [0:07] don't fully understand ourselves. [0:09] >> Jamie and Matel, thank you so much for [0:11] joining us. [0:11] >> I think we're going to have sex robots, [0:14] robots doing things that humans want and [0:17] that's going to be crazy in the [0:19] beginning and normalized over time. I [0:22] think people who are using AI and not [0:24] clearly articulating how they are using [0:26] AI are committing fraud. 100 years ago, [0:28] the Italian futurists had this [0:30] philosophy, forget the past. And it was [0:33] not coincidental that Italian futurism [0:36] morphed into fascism. The future is [0:39] going to be wonderful. It's going to [0:40] create all this prosperity. There's [0:42] sometimes a J curve, if you will. Things [0:44] go wrong for a couple of years. That [0:45] could be a really big problem. How do we [0:47] get past a potential dip? [0:49] >> We don't have the option of turning this [0:51] off. If we did, it would be a terrible [0:54] idea. We need to make sure that this [0:56] transition is happening with everybody [0:58] but not to everybody. [1:02] Before you skip forward, I invite you to [1:04] become a member of the rational optimist [1:06] society. It's completely free. All you [1:09] need to do is go to rational [1:11] optimistocciety.substack.com [1:14] and enter your email address. You'll [1:16] receive our weekly diary that details [1:18] the most exciting companies in Frontier [1:20] Technology today. So please enjoy this [1:22] conversation with futurist founder and [1:24] author Jamie Metsel. [1:27] My guest today has spent 25 years where [1:30] biology, geopolitics, and AI all [1:33] intersect. Drafting policy at the [1:36] Clinton White House, sitting on the [1:37] WHO's committee on human genome editing, [1:40] and writing books that defined how we [1:43] talk about hacking our own DNA. He [1:46] recently did something that nobody has [1:48] done before. He published the first [1:51] major non-fiction book to credit AI as a [1:54] co-author, the AI ten commandments. [1:57] Jamie Metsel, welcome to the Rational [1:59] Optimist podcast. Thank you so much for [2:01] joining us. [2:02] >> Thank you so much, Stephen. It's really [2:04] a pleasure and honor to be with you. [2:05] >> Jamie, before we get to your book, I [2:07] want to talk about you on this show. [2:10] We're obsessed about how people get so [2:13] much done, how people do these amazing [2:15] feats of accomplishment. Uh, I think I [2:18] believe you've done 60 marathons, a [2:20] dozen iron mans, 40 odd ultramarathons. [2:22] You have a PhD, you've written six [2:25] books. How the hell did you get so much [2:27] done? Seven books. How How the hell did [2:28] you get so much done? [2:30] >> Well, I could give you a long answer [2:33] that would talk about efficiency and [2:37] focus and really thinking about what you [2:40] want to do before you get rolling uh [2:42] doing it. Um, but that would all be [2:44] wrong because I'm 57 years old and I [2:49] think 50% of it uh is that I don't have [2:51] any kids. And when I look at people who [2:55] have kids, which is a magnificent thing [2:57] to do, that seems to take so much time. [3:00] I'm blown away by how people raise one [3:03] kid, let alone two kids. And I've I've [3:06] heard um that you have to feed them [3:08] every single day without fail. Jenny, if [3:12] my career goes horribly wrong, I'm going [3:14] to blame it on my three kids. [3:16] >> Yeah, 100%. You should tell them every [3:19] single day. Do you know what I've [3:21] sacrificed for you? But, you know, I [3:24] think that you're every one of those [3:26] kids is like writing a hundred books. [3:29] Uh, every one of the kids that you're [3:31] putting your ideas and your love and [3:33] your care. Uh, it's really fantastic. [3:36] And and congratulations to you for [3:38] having three kids. I'm sure they're [3:40] great kids and I am hoping that they are [3:43] wild and crazy optimists on the [3:47] realistic side of optimism. Um, look, [3:49] this show is called the rational [3:51] optimist. I I read your book. I thought [3:54] it was fascinating. Obviously, AI is the [3:56] biggest thing in the world happening [3:58] right now. I want you to set the table [4:01] for us. What does earned cleareyed [4:03] optimism, cleareyed rational optimism [4:06] about AI actually look like for you? On [4:09] one hand, we need to imagine all of the [4:13] areas where AI and the related supercon [4:15] converging uh using the title of my book [4:18] technologies can can lead us. And in [4:20] that book, I talk about the really [4:23] exciting future of healthcare, [4:25] agriculture, energy, advanced materials, [4:28] data storage. And it's really exciting. [4:30] my own father after my late father after [4:33] his cancer diagnosis, I kind of took [4:36] over working with his oncologist and we [4:38] had a a very unconventional [4:41] treatment uh plan based on the [4:43] principles that I was writing about at [4:45] the time in in the book and he had a [4:48] one-year life expectancy and we extended [4:50] it to three and we filled that time with [4:52] love and joy and meaning and happiness [4:55] and I had a big feature in AARP magazine [4:59] uh on this and an action plan for what [5:01] everybody can do. So, we should be [5:03] really excited and I think a lot of [5:06] people now because there's been this [5:07] quick turn against emotional turn [5:10] against AI, I think people are are many [5:13] people are are losing their connection [5:15] to the idea, the possibility, the wonder [5:18] of what these technologies can do. But [5:22] at the same time, if we are just blinded [5:24] optimism optimists, that's actually [5:27] going to be quite dangerous. There's a [5:29] reason why we have fear and anxiety and [5:33] why evolution has preserved those very [5:37] human emotions. If those were just [5:39] wasted emotions, probably they would [5:41] have been selected out over the course [5:43] of billions of years. But our fears and [5:45] our anxieties are our way of telling [5:49] ourselves that actually if we don't take [5:52] the right steps, if we don't do [5:53] everything we can to optimize potential [5:55] benefits and minimize potential harms, [5:58] then our worst fears can be realized. [6:00] The AI revolution could do a lot of [6:03] harm. Whether that's in warfare or [6:05] employment or in dehumanization [6:09] or deskkillification of of humans, there [6:12] are a lot of very dangerous downsides. [6:14] And so being an optimism, an optimist [6:17] and a a rational optimist, it requires [6:20] imagining what's the best case scenario. [6:23] And it requires imagining what's the [6:26] worstc case scenario. And it imagines [6:28] everybody on every level, whether it's [6:30] the individual level or family level or [6:32] corporate level, national, [6:34] international, whatever, coming up with [6:36] a a plan, a set of principles for here's [6:39] how I'm going to optimize the stuff I [6:41] want, and here's how I'm going to work [6:43] to minimize and hopefully prevent most [6:46] of the things that I that I don't want [6:48] to see. And no one alone can win this [6:51] battle, but together, what we can do is [6:54] drive toward a better world. And if you [6:56] if people doubt that, you can just look [6:58] at our whole history. We've had massive [7:01] downs and we've had huge abuses that [7:03] have come very significantly connected [7:06] to to technological innovations. But if [7:09] you follow the Matt Ridley principle or [7:12] the Steven Pinker principle and you look [7:15] at things from a broader perspective, [7:18] you can see that things are are getting [7:20] better over time. But that doesn't mean [7:23] there are big dips. The black plague was [7:26] a big dip. World War II was a big dip. [7:30] And so those big dips, even if things [7:31] are getting better, they're not going to [7:33] get better on their own. It's that's the [7:35] hard work of being a rational, committed [7:38] optimist. [7:39] >> I've talked to people like you and I [7:41] that are rational optimists about AI and [7:43] they say, "Hey, the future's going to be [7:46] wonderful. It's going to create all this [7:47] prosperity, all these jobs." But there's [7:49] a, as you said, there's there's [7:51] sometimes a dip, a J curve, if you will, [7:53] that things go wrong for a couple of [7:55] years, and it seems like in this day and [7:57] age, things going wrong for one or two [8:00] or five years or 10 years, that could be [8:02] a really, really big problem. So, how in [8:05] your view do we do we mitigate that? [8:07] Future could be awesome. It's going to [8:08] create all this amazing stuff, but how [8:10] do we how do we get past a potential [8:12] dip? Yeah, I think it's a very very [8:15] important question because there's the [8:16] dip in the sense that we are going to [8:19] have significant changes in employment [8:21] with this uh this technological [8:24] quantum leap as we have had in other [8:27] technological quantum leaps. All of our [8:29] ancestors were hunter gatherers and then [8:32] more and more of them became farmers and [8:34] then people left the farms uh through [8:36] industrialization and did all of these [8:38] other jobs. Uh but those things have [8:41] happened very gradually. Uh this change [8:44] is happening very quickly. I was just [8:46] the other day speaking with a very close [8:48] friend of mine who's a real estate [8:50] developer and he has about 50,000 uh [8:53] apartments. He owns the buildings and [8:55] and rents them out. And he was he was [8:57] saying that that this is going to mean [8:59] that he just needs a smaller headcount. [9:02] And exactly as you said, there will be [9:06] other entire industries that we can't [9:09] even imagine. I mean, you and I are [9:10] having this conversation remotely as a [9:14] result of magnificent technological [9:17] revolutions that made nobody 20 years [9:21] ago would have imagined podcasting as a [9:23] thing, let alone remote video uh [9:27] podcasting. And so there'll be whole [9:29] other um industries and and we can begin [9:33] imagining them and I do actually about [9:35] uh world building. I I'm also a novelist [9:38] and I just think that the the one of the [9:40] next generations of novels will be kind [9:44] of like video games where you'll have [9:46] maybe great novelists and everybody gets [9:48] one character and you build out a whole [9:51] backstory and a set of responses and [9:54] just the character uh of your the [9:57] character of your character that goes [10:00] into this worldbuing novel. And maybe [10:03] there's a central theme like kind of [10:05] like Grand Theft Auto. And then people [10:08] who are the equivalent of the readers of [10:10] novels can wander through those worlds [10:14] as they maybe tell a story like like [10:17] here in New York we have a a um a [10:19] theatrical installation called Sleep No [10:22] More where people wander through the [10:25] theater. And so it's so we that's just [10:28] one idea. there will be thousands uh [10:31] millions of new kind of crazy ideas that [10:34] now seem nuts that will just become so [10:38] normal. And coming back to your question [10:40] though, there is pain and will be pain [10:44] in this transition. And even if the [10:45] benefits are societal, that doesn't mean [10:48] which is the same with every other [10:50] transition that we've ever had that all [10:52] of the the losses will be evenly [10:56] distributed. And that's why I believe [10:59] societies have a massive obligation. Uh [11:03] some people saying everything I said [11:06] would say that's why I believe in [11:08] universal basic income. That's not where [11:10] I where I head. I believe in universal [11:12] basic services, health care and [11:14] education and safe housing and safety [11:17] and security and all of those those [11:19] things. I I don't think universal basic [11:22] income is uh is the answer. And I think [11:25] governments need to invest through [11:27] unemployment insurance, through [11:30] retraining, uh through facilitation. And [11:33] we talked about these new industries [11:35] giving small grants to help get a [11:38] thousand a million new ideas off of the [11:40] ground. Some of which will work and some [11:43] won't work. And and finally, what I'll [11:45] say is because I'm a humanist, I do not [11:49] believe I people talk about artificial [11:51] general intelligence, AGI, which is a [11:54] time when AI systems will be able to do [11:57] everything that humans can do. I have a [12:00] sevenletter response to that. AGI is BS. [12:03] I think AGI, the concept of AGI is a [12:06] bunch of baloney. uh that humans are [12:09] this magical species with all of these [12:12] skills and capabilities that we don't [12:14] fully understand ourselves. And I think [12:15] that we're going to find that while [12:17] machines and AI systems can do some [12:19] remarkable things, uh there'll be a lot [12:23] of things that machines can't do. And it [12:26] how wonderful to have as many humans as [12:29] possible doing highest level premium [12:33] human activities. But there are a lot of [12:34] people who are for example truck drivers [12:38] um or street cleaners um and those jobs [12:40] are going to change. We'll still have [12:42] humans uh but the jobs are going to [12:44] change and and the more that we can [12:46] facilitate in a kind and loving and [12:50] gracious and helpful way this transition [12:52] the better off we'll be. We don't have [12:54] the option of turning this off. If we [12:56] did, we it would be a terrible idea to [12:59] do it, but we need to make sure that the [13:02] this transition is as human centered as [13:05] as possible, that it's happening with [13:08] everybody, but not to everybody. [13:10] >> My my friend Tyler Cowan likes to frame [13:12] things as what rises in status versus [13:14] falls in status or or rises in value [13:16] versus falls in value, zoom out in seven [13:19] years, zoom out in their 10 years. what [13:22] skills, what industries do you think are [13:24] irreducibly human? Um what what core [13:28] values, what core tasks can AI never [13:30] touch? Another way of framing this is I [13:32] guess if if you had uh my three kids [13:35] sitting in front of you right now, what [13:36] would you tell them to to go do? [13:38] >> And so what I would say is I think this [13:40] is going to happen at the task level, [13:42] not at the industry level. That in every [13:45] industry there are going to be human [13:47] functions and machine functions. And and [13:50] what we're have going to have to do is [13:52] to break down not just every industry, [13:54] but every job, whatever job you have, [13:57] let's say you can break it down into 10 [14:00] tasks that are the core tasks that you [14:02] do for your job. And for every one of [14:05] those tasks, you can put it on a [14:07] spectrum. um on one side of the spectrum [14:10] is 5 or 10 years from now entirely a [14:13] machine function and another side of the [14:15] spectrum is entirely a human function or [14:18] else somewhere in between. If your job [14:21] when you go through it, all of those 10 [14:23] tasks are on the machine side, that [14:27] means that your job is probably going to [14:29] go away. And for you, the best thing is [14:32] starting now to think about about [14:34] transition because the last thing that [14:36] we want is humans functioning as second [14:39] rate machines. But in many kinds of [14:43] jobs, there are going to be some machine [14:45] tasks and some human tasks. And so for [14:48] the machine tasks, the mission is to [14:51] say, well, how can we best have machines [14:53] doing those tasks as quickly and well as [14:56] possible and not betting the store all [14:58] at once, but through a a bunch of [15:00] discrete examples uh of using these [15:03] systems to solve specific definable [15:06] tasks? And how can we overindex on [15:09] helping humans become the best possible [15:12] humans? And if it's somewhere in [15:13] between, thinking about that [15:15] relationship between humans and uh and [15:18] machines. And I I for one, I don't think [15:21] collectively there's going to be a job [15:24] apocalypse, but I do think there will be [15:27] some pretty big jobs that will be [15:30] significantly reduced in numbers. And so [15:33] coming to your kids, what I would say is [15:36] to really be always thinking and and [15:38] they're lucky to have you as their [15:40] father who is always thinking about well [15:42] where are we heading? And so we have to [15:45] live I I write non-fiction and I write [15:47] science fiction. And the reason I write [15:49] science fiction is that the world is [15:52] changing so rapidly that in order to [15:54] think realistically [15:56] about what's coming, uh we have to think [15:59] kind of like science fiction writers [16:00] because our brains have evolved in in a [16:04] very uh linear way, a very practical way [16:07] which has served our survival uh for for [16:11] basically ever. But now we need to be p [16:14] somewhat practical but we need to really [16:17] imagine what's coming and even if [16:20] whatever your whatever your job is going [16:22] from cook to janitor to CEO to author to [16:27] futurist uh just we really need to think [16:30] about what's coming and really [16:33] importantly we need to think about how [16:36] much we value everything up to this [16:38] point. 100 years ago, the Italian [16:40] futurists had this philosophy. Forget [16:42] the past. This is all about the [16:45] mechanized future. And it was not [16:47] coincidental that Italian futurism [16:49] morphed into fascism. We have this [16:53] wonderful human tradition. Our cultural [16:56] evolution over thousands of years, [17:00] longer than thousands of uh of years is [17:03] spectacular. uh and and the these AI [17:06] systems aren't this isn't wisdom coming [17:09] from Mars. This is the collection of our [17:15] cultural inheritance of all of our [17:17] culture. And so there's this othering of [17:19] AI. And what we really need to do is [17:21] invest in ourselves. And so for your [17:23] kids, I give a lot of talks both to [17:26] companies and to uh education leaders. [17:29] And I say that part of the future [17:32] involves being as literate as possible [17:35] in technological innovation and what we [17:38] can do and what it means. But another [17:41] part of it is turning everything off. [17:44] Every single person, whether you're a [17:46] kid, an adult, a professional, has to [17:50] have some part of your life totally [17:52] separate from all technology where you [17:55] are reconnecting [17:57] with you as a human, with the people [17:59] around you as a uh as a human. And this [18:03] isn't some kind of old old-fashioned [18:06] um idea. Uh, I think that there is so [18:10] much in humans that if we allow [18:12] ourselves to be so distracted by all of [18:15] these technologies, we're going to [18:18] actually become less. Not just less as [18:20] humans, but less effective at building a [18:24] kind of future that we'd all like to [18:26] inhabit. [18:27] >> Jimmy, I want to talk about your book, [18:30] the the the AI ten commandments. [18:32] Fascinating book. You put GPT5 on the [18:35] cover as a co-author. I'd love you to [18:38] take us inside the actual collaboration, [18:40] how you wrote a book with AI would have [18:43] this been possible with the original [18:45] chat GPT. Tell me everything about the [18:47] uh the process. [18:48] >> Yeah, thank you so much for asking, [18:50] Stephen. So, it all started in my Walden [18:53] Pond uh in the sense that two uh summers [18:57] ago, two years ago, I was invited to [18:59] this magical place in upstate New York [19:02] called the Shiitakqua Institution, which [19:04] is like a whole city that comes to life [19:06] for 10 weeks in the summer. And they [19:09] have music school and theater and an [19:12] opera. And there's a big very famous [19:15] lecture series in their in their famous [19:18] 6,000 person outdoor amphitheater where [19:20] six US presidents have spoken. So two [19:23] years ago I was invited to give the lead [19:26] keynote speech for the summer on the [19:28] future of AI. I did it. I got a 6,000 [19:31] person standing ovation. They invited me [19:33] back for the next summer and they said [19:35] speak about anything you want. And I [19:38] just I thought a lot about it and what I [19:40] decided was to connect my views on the [19:43] future of AI and other technologies and [19:45] the history of Shiakqua which is [19:48] 150year-old institution. It started out [19:51] as a place for Methodist Sunday school [19:53] teachers but it grew into a center for [19:55] religious pluralism. And so I wanted to [19:58] give a talk on AI and spirituality. [20:01] And I did my Walden pond. I just, you [20:04] know, I I went to the park and I just [20:06] really reflected without any technology [20:10] on what I wanted to say and then I put [20:13] my thoughts together into an outline for [20:16] that talk and I gave the talk and the [20:18] talk was extremely wellreceived and in [20:20] it I talked about the the intimate [20:22] connection between our technological [20:25] innovations and all of our religious and [20:28] spiritual traditions, what we call our [20:30] our Abrahamic traditions, our world our [20:32] world religions are all agricultural [20:35] faiths deeply connected to agriculture. [20:38] Protestantism is unimaginable [20:41] separate from the the printing press. So [20:44] we we have that is our history and it [20:45] what I said in that talk is that it's [20:47] inevitable that AI will have significant [20:51] implications for our religious and and [20:53] spiritual lives. And then the question [20:55] is how? And I went through a lot of [20:57] different uh examples. And then I went [21:00] through the uh biblical ten commandments [21:03] uh one by one as written. And and when [21:06] you read them literally uh nobody would [21:10] agree literally to almost all you know [21:13] maybe eight of the of the ten [21:16] commandments. It's like yeah you [21:17] shouldn't murder or sure you shouldn't [21:19] kill unless you're landing on the [21:22] beaches in Normandy. and then you know [21:24] kill for good reason because you're [21:26] you're you're ending the the you're [21:28] defeating the Nazis and ending the [21:30] ending the Holocaust. [21:31] it it's complicated. And so in that talk [21:34] I talked about my process of going to [21:38] then uh GBT4 and ask and I really [21:41] thought a lot about this uh this prompt [21:44] um was asking based on an assessment of [21:49] the entirety of human recorded history [21:51] and all of our different religious, [21:53] spiritual, moral and ethical traditions, [21:55] what are 10 principles that if followed [21:58] by everyone would lead to the greatest [22:00] amounts of peace, happiness and and [22:02] flourishing and it gave these beautiful [22:05] AI ten commandments based on on us not [22:09] based on AI's independent wisdom and [22:12] then there I had thousands of back and [22:14] forths interrogations [22:16] with AI with the AI saying well where's [22:19] this coming from and we dug and it it [22:21] was so beautiful because it was coming [22:23] from all of our traditions and all of [22:25] our traditions are all trying to do the [22:27] same thing to live moral ethical lives [22:30] to try to find a right balance balance [22:32] between the individual and and communal [22:34] needs. And there were but not just the [22:38] ones that are more familiar, but also [22:40] these more obscure uh traditions, [22:43] indigenous traditions from Africa and [22:45] the Americas and elsewhere and it was [22:47] really beautiful. So I gave that talk [22:50] and then as I was walking around the [22:52] campus, it's mostly older people. The [22:54] old people kind of attacked me and they [22:56] said, "Give us the AI ten commandments. [22:59] They are so beautiful. we want we want [23:01] them. And uh so that was and I said, you [23:04] know, I'm going to write a book. And so [23:06] when I decided to write a book, I knew I [23:09] had to write it differently because I [23:11] was writing about in at least in this [23:14] part with AI Ten Commandments, a [23:16] collaboration with AI. I mean, I I have [23:19] a PhD in history and I do a a ton of [23:24] reading, but I don't know everything [23:27] about every tradition in all of of the [23:30] world. And and no human could. It's just [23:32] it's beyond our our grasp. And so what I [23:35] did is I went back to metaphorical [23:37] Walden Pond and I uh put together the [23:40] oldfashioned way a very detailed outline [23:43] of the book on my own. And then I went [23:45] through the outline and I said found [23:48] specific areas where maybe there were a [23:51] few paragraphs where I was summarizing a [23:53] vast field of knowledge. And this is a [23:56] real example. I had a thesis statement. [23:58] AI has the potential to see us [24:00] collectively kind of like we have when [24:03] looking down from above at an ant [24:05] colony. And so then I trained GBT4 and [24:08] then five on my writing style by [24:11] uploading a bunch of my favorite [24:13] writing. And then I said I so I started [24:16] writing the book. And when I would get [24:17] to one of these sections where I wanted [24:19] to have this kind of insert, I would say [24:22] to GPT5, here's the thesis statement. [24:26] Write me three paragraphs in my writing [24:29] style making these five points. And it [24:31] would give me something that would be no [24:32] good. And then I'd say, "Not quite [24:35] right. Make these five changes. And not [24:37] quite right. Make these changes." So a [24:39] bunch of back and forth. And then there [24:40] would be something that the argument was [24:42] pretty good, but the writing wouldn't be [24:44] very good. And then I would take it out, [24:46] put it into Microsoft Word. I do a full [24:48] edit of those three paragraphs. Then I [24:50] would put it back into GPT5 and say, [24:53] "Well, how can how can I make these [24:55] three paragraphs better?" And it might [24:57] say, "You're using this word twice or [25:00] this sentence is a runon or or [25:02] whatever." So then there'd be a bunch of [25:03] back and forth. And then when it was [25:05] kind of good enough, I would put it in [25:07] the text and I would keep going. And so [25:09] that's how I did the first draft. Then I [25:11] went through I cut 40% of everything [25:15] because and then I pretty much rewrote [25:20] the entire manuscript and that took a [25:23] lot of time. Then I hired a phenomenal [25:26] human editor and we did a very very deep [25:30] frankly edits as as people who've [25:32] written books know are very painful [25:33] because you think well I'm almost done [25:35] and you're not almost done and it just a [25:38] huge edit because you know I'm I'm a [25:41] novelist as I said I I want every word [25:43] to count and so that was where the the [25:47] book came from and then when I was [25:50] deciding like the about the publishing [25:53] I just thought, well, because of the [25:56] nature of this collaboration, if I put [25:59] my name alone on the cover, I feel like [26:02] that would be fraud. If I had like a [26:05] genius, [26:07] probably spectrumy human who was my [26:10] writing partner and we interacted in all [26:12] of this way. Let's just say that this [26:14] spectrum human had actually my my friend [26:18] AJ Jacobs has in fact read the entire [26:21] encyclopedia. But let's just say this [26:23] spectrum human had read every book in [26:26] the world. And um but if I had that kind [26:29] of collaboration, I would of course put [26:32] that person's name alongside mine on the [26:34] cover of the book. So, it just felt like [26:36] it would be fraudulent for me to to not [26:40] put AI GB5 as my named co-author. Now [26:44] it's I I'm actually running into some [26:46] headwinds on this because the public has [26:50] very rapidly in many ways turned against [26:53] AI and people have a hard time [26:56] differentiating what I've just described [26:58] which I feel was very um creative and [27:02] collaborative and additive um and [27:05] something that I couldn't have done [27:06] alone and the AI couldn't have done [27:09] alone. So that way I think it's special [27:10] and people have a hard time [27:12] differentiating that and from your high [27:14] school kid you know cheating on their [27:17] essay by just printing out something [27:19] from chat GPT. And so that's it's been a [27:21] little bit of a challenge getting the [27:23] word out. Everybody who reads it then [27:25] they they get it. So that's it's been [27:28] very interesting to navigate what nine [27:30] months ago there was so much optimism [27:32] and the idea of collaborating with AI on [27:36] a book. the idea of using AI in a unique [27:39] way to mine the entirety of human uh [27:42] recorded history was exciting to people [27:44] and just very very quickly the public [27:47] mood has turned and there's so much fear [27:49] and anxiety and I hope that this book [27:51] can be a statement like well I'm not [27:54] discounting your fears but let's look at [27:56] what AI can do in you know in this area [28:00] and in other areas [28:02] >> having being you're you're already an [28:04] accomplished writer and have been for [28:06] for many many years. What advice would [28:09] you have for someone who is riding with [28:12] AI either themselves an accomplished [28:14] rider or just starting off? Where is the [28:16] line between what's right and what's [28:19] wrong? Because you can you can obviously [28:20] take this too far as well. [28:22] >> Yeah. [28:22] >> So what I would say is first start [28:25] without AI. The first and most important [28:28] thing is learn your voice. If you're [28:31] collaborating with AI, if GBT5 is your [28:33] voice, you have no voice because GBD5 is [28:37] a and and all these AI systems are [28:39] averagers. So, you have to know your [28:42] voice. And I think there's a real danger [28:45] uh for younger people now that they're [28:47] not going to have the space to find out [28:50] who they are intellectually, who they [28:52] are, what their voice sounds like. uh in [28:55] part because we have AI systems that can [28:57] do things for us and in part because [29:00] people have these digital systems on all [29:03] the time that are just pinging your [29:05] heads and just disrupting everything and [29:08] not giving people that that space. So [29:10] that's number number one. Number two is [29:13] however you are using uh AI, I'm for [29:18] radical transparency. I think people who [29:20] are using AI and not clearly [29:23] articulating how they are using AI in my [29:26] view are are committing fraud. I think [29:27] every scientific paper should now have a [29:30] section on the bottom not necessarily [29:33] saying I didn't use AI in any of this [29:36] but saying here's exactly how I used um [29:40] used AI but I have a novel coming out in [29:43] February called Virtuosa [29:46] which is about the intersection of AI [29:47] robotics and classical music and in that [29:51] novel one of the characters is a robot a [29:54] a next next next generation robot that's [29:57] able to do miraculous things with with [30:00] music. And so in a novel, if I have a [30:03] character that has that's doing [30:05] something that I don't do, you I'll go [30:07] if I write I'm writing about a beekeeper [30:09] and I'm not a beekeeper, I'll go and [30:11] I'll find some beekeepers and I'll spend [30:13] time with them and I'll ask them about [30:15] their lives and and maybe ask them role [30:18] plays of what would they do in a certain [30:21] situation. So that's what I did with [30:24] this character. I uh I wrote up the [30:26] character and then I entered this text [30:29] into chat GPT saying this is background [30:32] for your character. Now, let's do a role [30:35] play and I'm going to ask you some [30:38] questions and respond from the from the [30:41] perspective of this character. And [30:42] again, it wasn't I didn't use this for [30:44] the novel. It was background to [30:46] understand that character. And we had [30:48] these wonderful conversations that that [30:51] when the book comes out, I'm I'm going [30:53] to post them all on my uh on my website. [30:57] And I'm the first science fiction writer [30:59] in all of history who when writing about [31:01] a robot is able to have a sustained [31:05] background conversation with a machine. [31:08] And so I think that's pretty incredible. [31:11] And and so that's why again I think that [31:13] there's there's a lot of fraud that is [31:15] happening now of people who are having [31:18] AI do stuff and putting their name on it [31:21] and there's a little window of time when [31:23] you can get away with it. Uh but I think [31:26] that the real the best use of AI is [31:29] going to be one people using it to do [31:32] great things. And I talked about my my [31:35] late father's cancer. I mean that was [31:37] AI. I don't have the ability to an [31:39] analyze a whole human genome on my own. [31:42] Nobody does. So we would expect doctors [31:45] to be using and healthcare providers to [31:47] be using AI tools to optimize outcomes. [31:51] And so I think that's it's great, but we [31:53] need to define what it is and what do we [31:56] want? What's the role for humans and [31:58] whether it's human health care providers [32:01] or artists and other creatives or or or [32:04] really anything. And then we need [32:06] radical transparency. And so that was [32:08] for me one of the reasons why I insisted [32:10] having GBT5 as my named co-author is I'm [32:13] a I wanted to be a pioneer in this kind [32:18] of radical transparency. [32:20] >> Of your 10 AI commandments, do you have [32:23] a favorite? [32:24] >> Oh, you know, I think probably the last [32:26] one to live with awe, gratitude, and [32:29] love because we are so small. uh the AI [32:33] ten commandments. It's kind of like the [32:35] Aremis 2 astronauts looking back on [32:38] Earth and when they look back they just [32:40] see this little ball of life in this [32:42] dark empty mostly [32:46] lifeless universe even if there is and I [32:48] think there probably is life someplace [32:50] else. And so there we should have like a [32:53] level of awe of of just where we are. [32:58] And and and I think that when we do that [33:01] can kind of awe and love and and respect [33:04] animate so many of the uh of the other [33:08] things. And so um but you I to tell you [33:11] the truth I kind of love them all. And [33:14] every person who I have read side by [33:17] side the as written biblical ten [33:20] commandments and the AI ten [33:22] commandments, I have yet to meet a [33:25] single person who says at least when [33:28] read literally the biblical ones make [33:31] more sense than the AI ones. and and [33:33] then to say, well, given that these AI [33:36] ten commandments are drawn on from all [33:38] of our traditions, it's kind of a a [33:41] wonderful message that we're all part of [33:43] this this same thing, not just the same [33:46] humanity, but this same experiment of [33:49] life in in, you know, 3.8 billion years [33:52] on this planet that connects us to each [33:55] other and and all of life. And I think [33:57] that that perspective um is something [34:00] that's really beautiful and inspiring. [34:04] >> I love it. A week or so ago, Pope Leo [34:06] released his fourth encyclical which is [34:08] Magnifica Humanitus. Uh as I understood [34:12] it, the premise was technology is never [34:15] moral. It was in large part about AI. As [34:19] I read your book, it was kind of making [34:21] the opposite case of that. So, if you [34:23] could sit down with Pope Leo, what's the [34:25] one question you would want to ask him [34:28] or the thing that you would want to tell [34:30] him about? [34:30] >> Yeah, [34:31] >> it's great. And I I actually did a long [34:34] blog post on the papal papal encyclical [34:38] um which maybe you can can link to in [34:40] the uh in the show notes. And I have [34:43] spoken at the Vatican. Uh, and as a [34:45] matter of fact, I had a I was supposed [34:47] to meet with Pope Francis, but I had to [34:50] fly home uh to my original home in [34:53] Kansas City. Now I live in New York [34:55] because my uh my then 83year-old [34:59] mother was having her bach mitzvah. And [35:01] so I had to choose it was between my [35:03] mother and Pope Francis. And I I chose [35:05] my mother. But if I would have the honor [35:08] of sitting down with Pope Leo, what I [35:10] would say is I really loved the [35:12] encyclical. I really loved this [35:15] aspiration which I fully share to say [35:18] that these technological innovations [35:21] must serve humanity. They must serve [35:25] human values. And that's not going to [35:27] happen on its own. It's going to happen [35:30] with a level of consciousness, with a [35:33] level of aspiration. uh because no [35:35] technology comes with its own built-in [35:37] value system. It's up to us to infuse [35:41] our best values including our most [35:43] ancient and cherished values into the [35:46] technologies that we are creating. But I [35:49] had a couple of critiques uh of that in [35:53] the in spite of my extreme praise and [35:56] that was I felt like there wasn't that [35:58] much in the encyclical about the [36:02] incredible positive story of AI and and [36:05] related technologies about what it can [36:09] do to in health care and making our [36:12] agriculture more uh more efficient and [36:15] productive so we can feed more people [36:18] better uh with fewer inputs of land and [36:22] energy and and fertilizer and other [36:25] things and not to mention uh the [36:27] terrible risk and and the terrible cost [36:29] in suffering and pandemic threat and [36:32] antibiotic resistance of industrial [36:35] animal agriculture. So that was my my [36:38] biggest thing was to say let's it's [36:43] great to focus on the dangers and the [36:45] values and I certainly agree with both [36:47] but it felt a little bit unbalanced that [36:51] it didn't say how exciting these are and [36:55] that it is in my view and and I think [36:57] even in the church's view it's it's all [37:00] of our responsibility like if we had [37:02] health care providers who were slowing [37:05] down in treating cancer [37:07] because AI happened to be a technology [37:10] that was facilitating cancer care. I [37:13] would see that as a harm. At the same [37:16] time, you mentioned in your introduction [37:18] of me that I was a member of the World [37:20] Health Organization expert advisory [37:22] committee on human genome editing. As a [37:24] matter of fact, I was invited to the [37:26] Vatican to talk about that and was [37:28] invited to to uh write a a paper for the [37:33] Pontipical Council of Culture which they [37:35] which they published. And so that human [37:38] genome editing was made possible through [37:41] these revolutionary tools. And although [37:44] it is my strong view that we will have [37:48] genome edited human embryos that will [37:50] become humans in the future and we will [37:53] want to have it and it will be highly [37:55] moral and ethical to do it. Particularly [37:58] if we can change a 2-year life of [38:01] somebody born with a terrible deadly [38:03] genetic disorder into a 90 or 100year [38:08] fully realized life of somebody who the [38:10] same person who just doesn't have that [38:13] that single harmful harmful mutation. [38:19] But our conclusion of our committee was [38:22] we're not ready for it now. And so that [38:24] what happened in China in 2018 and 2019 [38:27] in my view was Nermberg style human [38:30] experimentation and I'm against that. [38:32] And so that's why I think there's this [38:33] danger to feel like the question being [38:36] asked is technology yes or no. And I [38:40] think that's really the wrong question. [38:41] The the question is technology how best? [38:44] And we have different perspectives. The [38:46] church is coming from one perspective. [38:48] I'm not a Catholic. I'm a I'm like many [38:51] others in in New York City. I'm a Jewish [38:54] atheist with Buddhist leanings. And so, [38:57] you know, I I you know, do I think that [39:00] the very traditional dictates of [39:03] Catholicism [39:04] um are are going to be our our core [39:08] guiding principles? No, I I do not. But [39:10] do I think that there are values not [39:13] just of the Catholic tradition but of [39:15] all traditions that can contribute to [39:19] letting our ethics guide our journey in [39:21] into this very radical future? I think [39:24] it's it's very important and that many [39:26] traditions have that and many like the [39:28] Catholic Church. I thought it was great [39:30] that the pope um used this opportunity [39:33] to sneak in an apology for the Catholic [39:36] Church condoning slavery and [39:39] colonialism. It was just a paragraph. It [39:41] didn't have to be there, but it was [39:43] clear that Pope Leo wanted to find a way [39:46] to say that. And so I thought that was [39:48] that was very positive. But I think that [39:50] it's the important thing is balancing [39:52] it. It's not if a mean this is rational [39:55] optimism. It's not blind optimism. Blind [39:58] optimism can actually be quite dangerous [40:02] and blind pessimism, overwhelming [40:04] pessimism in my in my view can be quite [40:07] dangerous. But rational optimism, [40:10] measured optimism, thoughtful optimism, [40:13] I think that's what's required. And I do [40:15] I do think actually that this encyclical [40:17] was very very different uh from the the [40:21] um Catholic Church treatment of [40:23] Capernacus or Galileo or frankly even of [40:26] stem cell research which I think the the [40:29] Catholic Church is is getting wrong and [40:31] I've been very public including at the [40:33] Vatican saying that. Um, but I think [40:36] it's very positive and I hope that it [40:38] inspires people from other traditions [40:41] and other backgrounds and other [40:43] organizations to start putting out their [40:45] principles. I thought that the the [40:47] anthropic AI constitution was helpful. [40:51] None of these things single-handedly are [40:53] going to be transformative, but [40:54] collectively I think they can be very uh [40:57] constructive and and additive. You have [40:59] a PhD in history. You also mentioned [41:02] there's the seed of a panic around AI [41:05] right now. There's a certain backlash [41:07] against the building of data centers, [41:09] what AI will do uh to jobs, all that [41:12] stuff. As you think back through your [41:14] knowledge of history, do you have a a [41:18] favorite or a a panic from throughout [41:20] history that most reminds you of what's [41:23] happening with AI right now? You know, [41:25] there is always a panic at every [41:29] transition. I'm sure that there was a [41:32] panic a million years ago when people [41:34] started to control fire. I'm sure there [41:36] was a panic uh 12,000 years ago with [41:40] agriculture when life lives and and [41:42] lifestyles [41:44] started to change. Obviously, we're [41:46] familiar with the with the lites uh with [41:50] with industrialization. I don't know if [41:52] I have a favorite, but I do think that [41:55] all of these fears are actually founded [41:59] that, you know, the people who were [42:01] worried about giving up their hunter [42:03] gathering nomadic lifestyles. I think [42:06] that was a legitimate concern actually [42:08] in for many thousands of years in [42:11] agriculture. the people who were doing [42:13] agriculture were were less healthy in [42:16] many many ways than their hunter [42:19] gatherer nomadic uh nomadic [42:22] counterparts. There were a lot of people [42:24] who were harmed by industrialization. [42:28] But when we look back actually [42:30] industrialization was really great. It's [42:33] not coincidental that industrialization [42:36] rose and slavery ended at roughly the [42:40] same time because humans we had a lot of [42:42] stuff that we wanted done and it [42:45] required power and whether that was [42:47] human power or animal power or machine [42:50] power we still needed the power. So all [42:54] of these these changes come with with [42:58] upsides and downsides. you you mentioned [43:00] in both super convergence and your new [43:03] book the AI ten commandments how I how [43:06] AI can help us do all these amazing [43:08] things the AI opportunities if you will [43:11] is there one specific breakthrough in [43:14] healthcare in medicine in education that [43:16] makes you really stand up and say wow [43:19] that's going to it's going to change the [43:20] world over the next 5 to 10 years yes so [43:24] in every one of those areas but probably [43:26] the one that I'm most excited about is [43:29] education. Uh you and I, Stephen, have [43:32] great educations. We come from places [43:34] where there's a a lot of opportunity, [43:36] but there's a whole lot of people who [43:39] don't have great educations, who are in [43:41] places where there isn't a lot of of [43:43] opportunity. And that doesn't just suck [43:46] for them. It sucks for us because we are [43:49] beneficiaries of this cultural [43:53] inheritance, this scientific [43:55] inheritance, which is an inheritance of [43:57] humanity. You and I are speaking now in [44:00] English a language which we didn't [44:03] invent but we have this wonderful [44:05] language and has all these words that [44:06] help us form our thoughts into these [44:09] sharable forms. And so if and young [44:13] people around the world now through [44:16] smartphones which many many people have [44:19] uh can access AI tutors along hopefully [44:23] alongside wise human tutors and wise [44:25] human elders who are helping guide and [44:28] and instill values and morals and and [44:31] ethics. I think we're going to unlock [44:34] such amazing human potential that is [44:38] going to benefit everybody and that so I [44:42] just couldn't be more excited. And so [44:44] for me the most exciting applications of [44:48] all of these technologies are about [44:51] helping the world's most vulnerable [44:53] people. And so yes, I'm excited about [44:55] the future of oncology. And yes, we're [44:58] already making incredible innovations [45:01] now with pancreatic cancer for example [45:03] that people thought well that was a long [45:05] a long way off. Alphafold is now in [45:09] pretty much every country in the world. [45:12] At last record there were something like [45:14] two million labs had used it to advance [45:18] their research. And that's incredibly [45:19] exciting. But what I'm most excited [45:21] about is all of these people in all of [45:24] these poor places and slums and and [45:28] remote poor agricultural communities. [45:31] How many Einsteins and Mozarts and [45:34] Beethovens are in those places? And so I [45:38] think to unlock that that human [45:40] potential [45:41] >> is just an unbelievable gift not just to [45:44] them but to all of us. I think these [45:45] technologies if we work in that [45:49] direction can can help make that happen. [45:51] >> Jimmy, this has been a truly fascinating [45:54] conversation. What I'd like to do is [45:55] close the podcast with a quick lightning [45:58] round, a game of overrated and [46:00] underrated. So, what I'm going to do is [46:02] I'm going to name 10 things that kind of [46:05] fall within your wheelhouse and you can [46:07] respond to each one of them. Overrated [46:09] or underrated. Great. [46:10] >> What for you? [46:11] >> Uh, let's let's roll them. [46:12] >> Okay. First one, AI agents [46:16] underrated. It's going to be such an [46:18] exciting thing. We're just at the [46:20] beginning of what AI systems can do and [46:22] people are talking about making hotel [46:25] reservations, but there's so much that [46:27] can happen in scientific research and [46:29] and so many other areas. There will be [46:31] downsides. Bad things will happen along [46:33] the way, but I think they're underrated [46:35] because I'm really excited about them. [46:38] >> China's AI capabilities. [46:40] >> I think it's underrated. I mean, China [46:42] is really focused. Uh they've put a lot [46:45] of energy into AI. They have a really [46:48] smart smart people. [46:51] They are not yet. China isn't yet at the [46:54] level uh certainly of of the United [46:56] States, but it's catching up pretty [46:59] quickly and it's focused on on catching [47:02] up. So, I think still underrated. [47:04] >> In the same vein, open- source models. [47:07] >> I think underrated. I mean, that's the [47:09] big gamble. China is using the same [47:13] strategy that Google used with Android [47:17] to break the Apple monopoly. So they [47:20] thought, well, we're not going to win [47:21] the foundation model race, but we're [47:23] going to use an open- source model. And [47:26] frankly, they they were beneficiaries [47:28] when Meta was also losing AI. They said, [47:32] "We're going to go open source." So the [47:34] second place person always goes open [47:36] source. Uh and there's a a lot of [47:39] opportunity. I I still believe in [47:40] foundation models. I think, you know, [47:42] our great companies are doing amazing [47:45] things. Uh and I'm not saying that the [47:47] future belongs to to open source. You [47:51] know, people, you know, Apple did pretty [47:53] well by creating a walled garden uh [47:57] ecosystem. Um but I think there's a a [48:00] long runway for open source [48:03] >> human genetic enhancement. [48:06] I think it again I this is this optimism [48:09] I think it's underrated. No, I think [48:10] it's underrated in the sense you people [48:13] believe now that we're capable. People's [48:16] like oh you should engineer your your [48:19] kids to play chess or it's really [48:22] complicated. We are at the very very [48:24] early stages of this. We understand such [48:28] a tiny fraction of the full complexity [48:30] of human biology. And so um but this is [48:34] a a fundamental technology. We're going [48:38] to use it more and more in animal [48:39] agriculture and plant agriculture and [48:41] that's already happening. Multipplex [48:44] edits will allow us to do more changes [48:47] at once. Um, but our ignorance, our [48:50] relative ignorance of what of genetics [48:53] make it so that if we're doing a lot of [48:55] changes, even if we correctly identify [48:58] that making these changes in a human [49:01] will lead toward these outcomes, we [49:03] don't know nearly enough about what the [49:06] potential downside risks because it's [49:08] not like one gene is doing one thing. [49:10] Uh, our bodies are very very complex and [49:13] interactive and we we we don't know what [49:15] we don't know, but we know that it's a [49:17] lot. So I think that people are [49:19] overestimating what's possible now. Uh [49:22] but at this rate of innovation, I think [49:24] exciting things will happen in in the [49:26] future. But I but I do think that when [49:29] when engineers and computer science [49:32] people talk about engineering biology, [49:35] they're thinking think thinking of it [49:37] like an engineering task. But those of [49:39] us who are enscconced in the biology [49:41] world know just how complicated biology [49:44] is and how how little we know. So, I [49:47] think it's it's going to be slower than [49:49] people anticipate, but I think [49:51] fundamentally transformative over time. [49:54] >> A Matt Ridley favorite of Amara's law on [49:56] that one. Maybe [49:57] >> Amara's law is you you overestimate the [49:59] short term and underestimate long term. [50:01] Yeah. Uh number five, humanoids. [50:04] Humanoid robots. [50:06] >> You know, I think um I seem to have just [50:08] the same answer. So, I think humanoid [50:10] robots are realistic. Humanoid robots [50:13] are a bit far away, farther away than [50:16] people seem uh people believe. But I [50:19] think we're going to have huge advances. [50:21] And my I mentioned my new novel Virtuosa [50:24] is all about this and and it's what it [50:26] feels like for humans when they [50:28] experience a humanoid robot that doesn't [50:30] look like a human, but it's a humanoid [50:32] robot that can do magical things with [50:34] classical music that is emotionally [50:36] resonant for humans. And I do think I do [50:39] believe that we are going to have AI [50:42] systems that can write their own code, [50:44] which is the case in my novel, that can [50:46] imagine goals and work toward creating [50:50] them. So I I I I think we're going to [50:52] have humanoid robots. They probably [50:54] aren't going to look like humans. I [50:55] mean, we I have an amazing uh robot in [50:58] my house. After dinner, I put my dishes [51:00] inside of this robot and add soap and [51:04] press a button. And so we we may have a [51:06] bunch of um of robots that are [51:10] structured for specific tasks, but we [51:14] live in a world that is organized around [51:16] our form. And so there there will be [51:18] cases where we want to have humanoid [51:20] robots. And I'll just say this as for [51:23] anybody who's uh who's listened all the [51:25] way in this in this podcast. Yeah. I I [51:28] think we're going to have sex robots. I [51:30] mean, we're going to have robots doing [51:32] things that humans want, and that's [51:36] going to be crazy in the beginning and [51:38] normalized over time, like IVF and kind [51:42] of everything else that we're scared of [51:43] and then becomes normal. [51:45] >> I'm going to end with two, Jamie. Yes. [51:47] First one, data centers in space, [51:51] >> you know, that I I don't I don't feel [51:53] like I know enough to fully make a bet. [51:56] I like the idea and there are good [51:59] reasons to put data centers in space. It [52:01] solves certainly one of the big problems [52:04] of cooling and we can get a lot of data [52:08] back and forth on on satellites. So in [52:11] principle I'm excited about it. Um but [52:15] it's one of those things where it's [52:18] going to be harder. I mean when you're [52:20] writing science fiction you can just [52:22] imagine it's there. But when you're [52:24] building the thing at scale and it needs [52:27] to be cost effective that that's [52:30] actually a big task. But I I think it it [52:33] could work and ought to work and maybe [52:34] not for everything but for certain tasks [52:37] and I think it it will be positive. [52:40] >> Last one which is very relevant to you [52:43] writing books in the age of AI so [52:47] important and so hard. I mean, I, as I [52:50] said before, being a human, doing core [52:55] human stuff is going to be more [52:58] important than ever before. And our [53:00] greatest artists who touch the core of [53:03] their humanity and our humanity will be [53:07] more important than ever before. And I [53:09] think societies are going to have to [53:11] really invest in making space for human [53:15] artists being humans. We also need to [53:18] make a space for human creators [53:20] collaborating with technologies in new [53:23] and innovative ways that will will blow [53:25] our mind. But we really the the human [53:28] artist, the human soul, human [53:30] imagination, human uh creativity is so [53:35] important in and of itself and for [53:38] building the best possible future and [53:42] for actually having technology that [53:44] serves human needs. Jamie Medsel, thank [53:47] you so much for taking the time to sit [53:49] down with us. I want to give you a [53:50] chance to tell everyone where they can [53:52] find out more about your book, where [53:53] they can buy the book, where they can [53:55] follow all your work. [53:56] >> Oh, thank you so much. Uh, so the [53:58] easiest way to get everything is through [53:59] my website, jammetzel.com. [54:02] jamm.com. [54:06] Don't put a second e in my last name. [54:08] People have been trying it for years and [54:10] it's wrong. Uh, and you can get the [54:12] book. I mean, there's there's links on [54:14] that site. You can get the books on on [54:15] really anywhere, Amazon, your um your [54:18] local bookstore. I have a newsletter, so [54:20] if you go to my site, you can sign up [54:22] for my free newsletter, and I'm happy to [54:25] share my thoughts, such as they are, [54:27] with with anyone who signs up. [54:30] >> Jamie, thanks so much for your time, and [54:31] I hope we can do it again soon. [54:33] >> Thank you so much, Stephen. It's really [54:34] been a pleasure.