---
title: 'AGI Is BS | Jamie Metzl''s AI Prediction'
source: 'https://youtube.com/watch?v=rlRxdFvK4aE'
video_id: 'rlRxdFvK4aE'
date: 2026-06-18
duration_sec: 3293
---

# AGI Is BS | Jamie Metzl's AI Prediction

> Source: [AGI Is BS | Jamie Metzl's AI Prediction](https://youtube.com/watch?v=rlRxdFvK4aE)

## Summary

Futurist and author Jamie Metzl discusses the hype around Artificial General Intelligence (AGI), calling the concept "baloney" and emphasizing that humans possess unique skills machines cannot replicate. He advocates for "rational optimism"—acknowledging both the promise and perils of AI—and stresses the need to focus on human-centered transitions, radical transparency, and investing in human potential. The conversation covers his book 'The AI Ten Commandments', co-authored with GPT-5, and explores AI's impact on jobs, education, healthcare, and ethics.

### Key Points

- **AGI is BS** [0:00] — Jamie Metzl asserts that the concept of AGI is a bunch of baloney, as humans are a magical species with skills and capabilities we don't fully understand ourselves.
- **Transparency and Fraud** [0:22] — He states that people using AI without clearly articulating how they are using it are committing fraud.
- **Historical Parallel: Italian Futurism** [0:28] — 100 years ago, Italian futurists had a philosophy to forget the past, which was not coincidentally connected to the rise of fascism.
- **Rational Optimism about AI** [3:58] — Metzl explains that being a rational optimist requires imagining both best-case and worst-case scenarios and creating plans to optimize benefits and minimize harms.
- **AI Revolution Dangers** [6:00] — He warns that the AI revolution could cause harm in warfare, employment, dehumanization, and deskillification of humans.
- **Historical Positive Trend with Dips** [8:27] — Despite massive downs and abuses tied to technological innovations, history shows things are getting better over time, but there are big dips like the Black Plague and World War II.
- **Job Transition and Pain** [9:02] — Metzl acknowledges significant changes in employment will occur quickly, causing pain, but new industries will emerge that cannot be imagined yet.
- **Universal Basic Services vs. UBI** [11:07] — He advocates for universal basic services (healthcare, education, safe housing) rather than universal basic income, and for government investment in retraining and facilitation.
- **Task-Level Change** [13:12] — AI change will happen at the task level, not industry level. Every job can be broken into tasks; some become machine tasks, some remain human tasks. The key is to help humans become the best possible humans.
- **Human Values and Cultural Inheritance** [16:38] — He warns against forgetting the past; AI systems are a collection of humanity's cultural inheritance, not wisdom from Mars. Invest in ourselves and maintain technology-free zones.
- **Writing a Book with AI Co-Author** [18:48] — Metzl describes his process: starting without AI to learn his voice, then using GPT-5 in an iterative, collaborative process, eventually editing comprehensively with a human editor. He credits GPT-5 as co-author for radical transparency.
- **AI Ten Commandments** [23:16] — He asked GPT-4/5: "Based on an assessment of the entirety of human recorded history and all our religious, spiritual, moral and ethical traditions, what are 10 principles that if followed by everyone would lead to the greatest amounts of peace, happiness and flourishing?" The result drew from all traditions, including obscure indigenous ones.
- **Favorite Commandment: Live with Awe, Gratitude, and Love** [30:20] — Metzl's favorite AI commandment is 'to live with awe, gratitude, and love' because it reminds us of our small place in the universe and animates other commandments.
- **Critique of Pope's Encyclical on AI** [34:04] — He praises the pope's encyclical for saying technology must serve humanity but criticizes it for lacking the positive story of AI in healthcare, agriculture, and education, and for focusing too much on dangers.
- **Historical Panics and AI** [41:25] — Metzl notes that every technological transition has caused panic (fire, agriculture, industrialization) and that those fears were legitimate but the long-term outcomes were positive.
- **Most Exciting AI Breakthrough: Education** [43:22] — He is most excited about AI in education, unlocking potential in vulnerable communities worldwide, potentially revealing 'Einsteins and Mozarts' in poor areas.
- **AI Agents: Underrated** [46:16] — Metzl believes AI agents are underrated because of their potential in scientific research and many other areas beyond simple tasks.

### Conclusion

The future with AI is neither purely utopian nor dystopian; it requires rational optimism, proactive planning, and a deep commitment to human values. By focusing on task-level adaptation, radical transparency, and human-centered transitions, we can navigate the challenges and unlock AI's enormous potential to benefit everyone.

## Transcript

concept of AGI is a bunch of baloney.
Humans are this magical species with all
of these skills and capabilities that we
don't fully understand ourselves.
>> Jamie and Matel, thank you so much for
joining us.
>> I think we're going to have sex robots,
robots doing things that humans want and
that's going to be crazy in the
beginning and normalized over time. I
think people who are using AI and not
clearly articulating how they are using
AI are committing fraud. 100 years ago,
the Italian futurists had this
philosophy, forget the past. And it was
not coincidental that Italian futurism
morphed into fascism. The future is
going to be wonderful. It's going to
create all this prosperity. There's
sometimes a J curve, if you will. Things
go wrong for a couple of years. That
could be a really big problem. How do we
get past a potential dip?
>> We don't have the option of turning this
off. If we did, it would be a terrible
idea. We need to make sure that this
transition is happening with everybody
but not to everybody.
Before you skip forward, I invite you to
become a member of the rational optimist
society. It's completely free. All you
need to do is go to rational
optimistocciety.substack.com
and enter your email address. You'll
receive our weekly diary that details
the most exciting companies in Frontier
Technology today. So please enjoy this
conversation with futurist founder and
author Jamie Metsel.
My guest today has spent 25 years where
biology, geopolitics, and AI all
intersect. Drafting policy at the
Clinton White House, sitting on the
WHO's committee on human genome editing,
and writing books that defined how we
talk about hacking our own DNA. He
recently did something that nobody has
done before. He published the first
major non-fiction book to credit AI as a
co-author, the AI ten commandments.
Jamie Metsel, welcome to the Rational
Optimist podcast. Thank you so much for
joining us.
>> Thank you so much, Stephen. It's really
a pleasure and honor to be with you.
>> Jamie, before we get to your book, I
want to talk about you on this show.
We're obsessed about how people get so
much done, how people do these amazing
feats of accomplishment. Uh, I think I
believe you've done 60 marathons, a
dozen iron mans, 40 odd ultramarathons.
You have a PhD, you've written six
books. How the hell did you get so much
done? Seven books. How How the hell did
you get so much done?
>> Well, I could give you a long answer
that would talk about efficiency and
focus and really thinking about what you
want to do before you get rolling uh
doing it. Um, but that would all be
wrong because I'm 57 years old and I
think 50% of it uh is that I don't have
any kids. And when I look at people who
have kids, which is a magnificent thing
to do, that seems to take so much time.
I'm blown away by how people raise one
kid, let alone two kids. And I've I've
heard um that you have to feed them
every single day without fail. Jenny, if
my career goes horribly wrong, I'm going
to blame it on my three kids.
>> Yeah, 100%. You should tell them every
single day. Do you know what I've
sacrificed for you? But, you know, I
think that you're every one of those
kids is like writing a hundred books.
Uh, every one of the kids that you're
putting your ideas and your love and
your care. Uh, it's really fantastic.
And and congratulations to you for
having three kids. I'm sure they're
great kids and I am hoping that they are
wild and crazy optimists on the
realistic side of optimism. Um, look,
this show is called the rational
optimist. I I read your book. I thought
it was fascinating. Obviously, AI is the
biggest thing in the world happening
right now. I want you to set the table
for us. What does earned cleareyed
optimism, cleareyed rational optimism
about AI actually look like for you? On
one hand, we need to imagine all of the
areas where AI and the related supercon
converging uh using the title of my book
technologies can can lead us. And in
that book, I talk about the really
exciting future of healthcare,
agriculture, energy, advanced materials,
data storage. And it's really exciting.
my own father after my late father after
his cancer diagnosis, I kind of took
over working with his oncologist and we
had a a very unconventional
treatment uh plan based on the
principles that I was writing about at
the time in in the book and he had a
one-year life expectancy and we extended
it to three and we filled that time with
love and joy and meaning and happiness
and I had a big feature in AARP magazine
uh on this and an action plan for what
everybody can do. So, we should be
really excited and I think a lot of
people now because there's been this
quick turn against emotional turn
against AI, I think people are are many
people are are losing their connection
to the idea, the possibility, the wonder
of what these technologies can do. But
at the same time, if we are just blinded
optimism optimists, that's actually
going to be quite dangerous. There's a
reason why we have fear and anxiety and
why evolution has preserved those very
human emotions. If those were just
wasted emotions, probably they would
have been selected out over the course
of billions of years. But our fears and
our anxieties are our way of telling
ourselves that actually if we don't take
the right steps, if we don't do
everything we can to optimize potential
benefits and minimize potential harms,
then our worst fears can be realized.
The AI revolution could do a lot of
harm. Whether that's in warfare or
employment or in dehumanization
or deskkillification of of humans, there
are a lot of very dangerous downsides.
And so being an optimism, an optimist
and a a rational optimist, it requires
imagining what's the best case scenario.
And it requires imagining what's the
worstc case scenario. And it imagines
everybody on every level, whether it's
the individual level or family level or
corporate level, national,
international, whatever, coming up with
a a plan, a set of principles for here's
how I'm going to optimize the stuff I
want, and here's how I'm going to work
to minimize and hopefully prevent most
of the things that I that I don't want
to see. And no one alone can win this
battle, but together, what we can do is
drive toward a better world. And if you
if people doubt that, you can just look
at our whole history. We've had massive
downs and we've had huge abuses that
have come very significantly connected
to to technological innovations. But if
you follow the Matt Ridley principle or
the Steven Pinker principle and you look
at things from a broader perspective,
you can see that things are are getting
better over time. But that doesn't mean
there are big dips. The black plague was
a big dip. World War II was a big dip.
And so those big dips, even if things
are getting better, they're not going to
get better on their own. It's that's the
hard work of being a rational, committed
optimist.
>> I've talked to people like you and I
that are rational optimists about AI and
they say, "Hey, the future's going to be
wonderful. It's going to create all this
prosperity, all these jobs." But there's
a, as you said, there's there's
sometimes a dip, a J curve, if you will,
that things go wrong for a couple of
years, and it seems like in this day and
age, things going wrong for one or two
or five years or 10 years, that could be
a really, really big problem. So, how in
your view do we do we mitigate that?
Future could be awesome. It's going to
create all this amazing stuff, but how
do we how do we get past a potential
dip? Yeah, I think it's a very very
important question because there's the
dip in the sense that we are going to
have significant changes in employment
with this uh this technological
quantum leap as we have had in other
technological quantum leaps. All of our
ancestors were hunter gatherers and then
more and more of them became farmers and
then people left the farms uh through
industrialization and did all of these
other jobs. Uh but those things have
happened very gradually. Uh this change
is happening very quickly. I was just
the other day speaking with a very close
friend of mine who's a real estate
developer and he has about 50,000 uh
apartments. He owns the buildings and
and rents them out. And he was he was
saying that that this is going to mean
that he just needs a smaller headcount.
And exactly as you said, there will be
other entire industries that we can't
even imagine. I mean, you and I are
having this conversation remotely as a
result of magnificent technological
revolutions that made nobody 20 years
ago would have imagined podcasting as a
thing, let alone remote video uh
podcasting. And so there'll be whole
other um industries and and we can begin
imagining them and I do actually about
uh world building. I I'm also a novelist
and I just think that the the one of the
next generations of novels will be kind
of like video games where you'll have
maybe great novelists and everybody gets
one character and you build out a whole
backstory and a set of responses and
just the character uh of your the
character of your character that goes
into this worldbuing novel. And maybe
there's a central theme like kind of
like Grand Theft Auto. And then people
who are the equivalent of the readers of
novels can wander through those worlds
as they maybe tell a story like like
here in New York we have a a um a
theatrical installation called Sleep No
More where people wander through the
theater. And so it's so we that's just
one idea. there will be thousands uh
millions of new kind of crazy ideas that
now seem nuts that will just become so
normal. And coming back to your question
though, there is pain and will be pain
in this transition. And even if the
benefits are societal, that doesn't mean
which is the same with every other
transition that we've ever had that all
of the the losses will be evenly
distributed. And that's why I believe
societies have a massive obligation. Uh
some people saying everything I said
would say that's why I believe in
universal basic income. That's not where
I where I head. I believe in universal
basic services, health care and
education and safe housing and safety
and security and all of those those
things. I I don't think universal basic
income is uh is the answer. And I think
governments need to invest through
unemployment insurance, through
retraining, uh through facilitation. And
we talked about these new industries
giving small grants to help get a
thousand a million new ideas off of the
ground. Some of which will work and some
won't work. And and finally, what I'll
say is because I'm a humanist, I do not
believe I people talk about artificial
general intelligence, AGI, which is a
time when AI systems will be able to do
everything that humans can do. I have a
sevenletter response to that. AGI is BS.
I think AGI, the concept of AGI is a
bunch of baloney. uh that humans are
this magical species with all of these
skills and capabilities that we don't
fully understand ourselves. And I think
that we're going to find that while
machines and AI systems can do some
remarkable things, uh there'll be a lot
of things that machines can't do. And it
how wonderful to have as many humans as
possible doing highest level premium
human activities. But there are a lot of
people who are for example truck drivers
um or street cleaners um and those jobs
are going to change. We'll still have
humans uh but the jobs are going to
change and and the more that we can
facilitate in a kind and loving and
gracious and helpful way this transition
the better off we'll be. We don't have
the option of turning this off. If we
did, we it would be a terrible idea to
do it, but we need to make sure that the
this transition is as human centered as
as possible, that it's happening with
everybody, but not to everybody.
>> My my friend Tyler Cowan likes to frame
things as what rises in status versus
falls in status or or rises in value
versus falls in value, zoom out in seven
years, zoom out in their 10 years. what
skills, what industries do you think are
irreducibly human? Um what what core
values, what core tasks can AI never
touch? Another way of framing this is I
guess if if you had uh my three kids
sitting in front of you right now, what
would you tell them to to go do?
>> And so what I would say is I think this
is going to happen at the task level,
not at the industry level. That in every
industry there are going to be human
functions and machine functions. And and
what we're have going to have to do is
to break down not just every industry,
but every job, whatever job you have,
let's say you can break it down into 10
tasks that are the core tasks that you
do for your job. And for every one of
those tasks, you can put it on a
spectrum. um on one side of the spectrum
is 5 or 10 years from now entirely a
machine function and another side of the
spectrum is entirely a human function or
else somewhere in between. If your job
when you go through it, all of those 10
tasks are on the machine side, that
means that your job is probably going to
go away. And for you, the best thing is
starting now to think about about
transition because the last thing that
we want is humans functioning as second
rate machines. But in many kinds of
jobs, there are going to be some machine
tasks and some human tasks. And so for
the machine tasks, the mission is to
say, well, how can we best have machines
doing those tasks as quickly and well as
possible and not betting the store all
at once, but through a a bunch of
discrete examples uh of using these
systems to solve specific definable
tasks? And how can we overindex on
helping humans become the best possible
humans? And if it's somewhere in
between, thinking about that
relationship between humans and uh and
machines. And I I for one, I don't think
collectively there's going to be a job
apocalypse, but I do think there will be
some pretty big jobs that will be
significantly reduced in numbers. And so
coming to your kids, what I would say is
to really be always thinking and and
they're lucky to have you as their
father who is always thinking about well
where are we heading? And so we have to
live I I write non-fiction and I write
science fiction. And the reason I write
science fiction is that the world is
changing so rapidly that in order to
think realistically
about what's coming, uh we have to think
kind of like science fiction writers
because our brains have evolved in in a
very uh linear way, a very practical way
which has served our survival uh for for
basically ever. But now we need to be p
somewhat practical but we need to really
imagine what's coming and even if
whatever your whatever your job is going
from cook to janitor to CEO to author to
futurist uh just we really need to think
about what's coming and really
importantly we need to think about how
much we value everything up to this
point. 100 years ago, the Italian
futurists had this philosophy. Forget
the past. This is all about the
mechanized future. And it was not
coincidental that Italian futurism
morphed into fascism. We have this
wonderful human tradition. Our cultural
evolution over thousands of years,
longer than thousands of uh of years is
spectacular. uh and and the these AI
systems aren't this isn't wisdom coming
from Mars. This is the collection of our
cultural inheritance of all of our
culture. And so there's this othering of
AI. And what we really need to do is
invest in ourselves. And so for your
kids, I give a lot of talks both to
companies and to uh education leaders.
And I say that part of the future
involves being as literate as possible
in technological innovation and what we
can do and what it means. But another
part of it is turning everything off.
Every single person, whether you're a
kid, an adult, a professional, has to
have some part of your life totally
separate from all technology where you
are reconnecting
with you as a human, with the people
around you as a uh as a human. And this
isn't some kind of old old-fashioned
um idea. Uh, I think that there is so
much in humans that if we allow
ourselves to be so distracted by all of
these technologies, we're going to
actually become less. Not just less as
humans, but less effective at building a
kind of future that we'd all like to
inhabit.
>> Jimmy, I want to talk about your book,
the the the AI ten commandments.
Fascinating book. You put GPT5 on the
cover as a co-author. I'd love you to
take us inside the actual collaboration,
how you wrote a book with AI would have
this been possible with the original
chat GPT. Tell me everything about the
uh the process.
>> Yeah, thank you so much for asking,
Stephen. So, it all started in my Walden
Pond uh in the sense that two uh summers
ago, two years ago, I was invited to
this magical place in upstate New York
called the Shiitakqua Institution, which
is like a whole city that comes to life
for 10 weeks in the summer. And they
have music school and theater and an
opera. And there's a big very famous
lecture series in their in their famous
6,000 person outdoor amphitheater where
six US presidents have spoken. So two
years ago I was invited to give the lead
keynote speech for the summer on the
future of AI. I did it. I got a 6,000
person standing ovation. They invited me
back for the next summer and they said
speak about anything you want. And I
just I thought a lot about it and what I
decided was to connect my views on the
future of AI and other technologies and
the history of Shiakqua which is
150year-old institution. It started out
as a place for Methodist Sunday school
teachers but it grew into a center for
religious pluralism. And so I wanted to
give a talk on AI and spirituality.
And I did my Walden pond. I just, you
know, I I went to the park and I just
really reflected without any technology
on what I wanted to say and then I put
my thoughts together into an outline for
that talk and I gave the talk and the
talk was extremely wellreceived and in
it I talked about the the intimate
connection between our technological
innovations and all of our religious and
spiritual traditions, what we call our
our Abrahamic traditions, our world our
world religions are all agricultural
faiths deeply connected to agriculture.
Protestantism is unimaginable
separate from the the printing press. So
we we have that is our history and it
what I said in that talk is that it's
inevitable that AI will have significant
implications for our religious and and
spiritual lives. And then the question
is how? And I went through a lot of
different uh examples. And then I went
through the uh biblical ten commandments
uh one by one as written. And and when
you read them literally uh nobody would
agree literally to almost all you know
maybe eight of the of the ten
commandments. It's like yeah you
shouldn't murder or sure you shouldn't
kill unless you're landing on the
beaches in Normandy. and then you know
kill for good reason because you're
you're you're ending the the you're
defeating the Nazis and ending the
ending the Holocaust.
it it's complicated. And so in that talk
I talked about my process of going to
then uh GBT4 and ask and I really
thought a lot about this uh this prompt
um was asking based on an assessment of
the entirety of human recorded history
and all of our different religious,
spiritual, moral and ethical traditions,
what are 10 principles that if followed
by everyone would lead to the greatest
amounts of peace, happiness and and
flourishing and it gave these beautiful
AI ten commandments based on on us not
based on AI's independent wisdom and
then there I had thousands of back and
forths interrogations
with AI with the AI saying well where's
this coming from and we dug and it it
was so beautiful because it was coming
from all of our traditions and all of
our traditions are all trying to do the
same thing to live moral ethical lives
to try to find a right balance balance
between the individual and and communal
needs. And there were but not just the
ones that are more familiar, but also
these more obscure uh traditions,
indigenous traditions from Africa and
the Americas and elsewhere and it was
really beautiful. So I gave that talk
and then as I was walking around the
campus, it's mostly older people. The
old people kind of attacked me and they
said, "Give us the AI ten commandments.
They are so beautiful. we want we want
them. And uh so that was and I said, you
know, I'm going to write a book. And so
when I decided to write a book, I knew I
had to write it differently because I
was writing about in at least in this
part with AI Ten Commandments, a
collaboration with AI. I mean, I I have
a PhD in history and I do a a ton of
reading, but I don't know everything
about every tradition in all of of the
world. And and no human could. It's just
it's beyond our our grasp. And so what I
did is I went back to metaphorical
Walden Pond and I uh put together the
oldfashioned way a very detailed outline
of the book on my own. And then I went
through the outline and I said found
specific areas where maybe there were a
few paragraphs where I was summarizing a
vast field of knowledge. And this is a
real example. I had a thesis statement.
AI has the potential to see us
collectively kind of like we have when
looking down from above at an ant
colony. And so then I trained GBT4 and
then five on my writing style by
uploading a bunch of my favorite
writing. And then I said I so I started
writing the book. And when I would get
to one of these sections where I wanted
to have this kind of insert, I would say
to GPT5, here's the thesis statement.
Write me three paragraphs in my writing
style making these five points. And it
would give me something that would be no
good. And then I'd say, "Not quite
right. Make these five changes. And not
quite right. Make these changes." So a
bunch of back and forth. And then there
would be something that the argument was
pretty good, but the writing wouldn't be
very good. And then I would take it out,
put it into Microsoft Word. I do a full
edit of those three paragraphs. Then I
would put it back into GPT5 and say,
"Well, how can how can I make these
three paragraphs better?" And it might
say, "You're using this word twice or
this sentence is a runon or or
whatever." So then there'd be a bunch of
back and forth. And then when it was
kind of good enough, I would put it in
the text and I would keep going. And so
that's how I did the first draft. Then I
went through I cut 40% of everything
because and then I pretty much rewrote
the entire manuscript and that took a
lot of time. Then I hired a phenomenal
human editor and we did a very very deep
frankly edits as as people who've
written books know are very painful
because you think well I'm almost done
and you're not almost done and it just a
huge edit because you know I'm I'm a
novelist as I said I I want every word
to count and so that was where the the
book came from and then when I was
deciding like the about the publishing
I just thought, well, because of the
nature of this collaboration, if I put
my name alone on the cover, I feel like
that would be fraud. If I had like a
genius,
probably spectrumy human who was my
writing partner and we interacted in all
of this way. Let's just say that this
spectrum human had actually my my friend
AJ Jacobs has in fact read the entire
encyclopedia. But let's just say this
spectrum human had read every book in
the world. And um but if I had that kind
of collaboration, I would of course put
that person's name alongside mine on the
cover of the book. So, it just felt like
it would be fraudulent for me to to not
put AI GB5 as my named co-author. Now
it's I I'm actually running into some
headwinds on this because the public has
very rapidly in many ways turned against
AI and people have a hard time
differentiating what I've just described
which I feel was very um creative and
collaborative and additive um and
something that I couldn't have done
alone and the AI couldn't have done
alone. So that way I think it's special
and people have a hard time
differentiating that and from your high
school kid you know cheating on their
essay by just printing out something
from chat GPT. And so that's it's been a
little bit of a challenge getting the
word out. Everybody who reads it then
they they get it. So that's it's been
very interesting to navigate what nine
months ago there was so much optimism
and the idea of collaborating with AI on
a book. the idea of using AI in a unique
way to mine the entirety of human uh
recorded history was exciting to people
and just very very quickly the public
mood has turned and there's so much fear
and anxiety and I hope that this book
can be a statement like well I'm not
discounting your fears but let's look at
what AI can do in you know in this area
and in other areas
>> having being you're you're already an
accomplished writer and have been for
for many many years. What advice would
you have for someone who is riding with
AI either themselves an accomplished
rider or just starting off? Where is the
line between what's right and what's
wrong? Because you can you can obviously
take this too far as well.
>> Yeah.
>> So what I would say is first start
without AI. The first and most important
thing is learn your voice. If you're
collaborating with AI, if GBT5 is your
voice, you have no voice because GBD5 is
a and and all these AI systems are
averagers. So, you have to know your
voice. And I think there's a real danger
uh for younger people now that they're
not going to have the space to find out
who they are intellectually, who they
are, what their voice sounds like. uh in
part because we have AI systems that can
do things for us and in part because
people have these digital systems on all
the time that are just pinging your
heads and just disrupting everything and
not giving people that that space. So
that's number number one. Number two is
however you are using uh AI, I'm for
radical transparency. I think people who
are using AI and not clearly
articulating how they are using AI in my
view are are committing fraud. I think
every scientific paper should now have a
section on the bottom not necessarily
saying I didn't use AI in any of this
but saying here's exactly how I used um
used AI but I have a novel coming out in
February called Virtuosa
which is about the intersection of AI
robotics and classical music and in that
novel one of the characters is a robot a
a next next next generation robot that's
able to do miraculous things with with
music. And so in a novel, if I have a
character that has that's doing
something that I don't do, you I'll go
if I write I'm writing about a beekeeper
and I'm not a beekeeper, I'll go and
I'll find some beekeepers and I'll spend
time with them and I'll ask them about
their lives and and maybe ask them role
plays of what would they do in a certain
situation. So that's what I did with
this character. I uh I wrote up the
character and then I entered this text
into chat GPT saying this is background
for your character. Now, let's do a role
play and I'm going to ask you some
questions and respond from the from the
perspective of this character. And
again, it wasn't I didn't use this for
the novel. It was background to
understand that character. And we had
these wonderful conversations that that
when the book comes out, I'm I'm going
to post them all on my uh on my website.
And I'm the first science fiction writer
in all of history who when writing about
a robot is able to have a sustained
background conversation with a machine.
And so I think that's pretty incredible.
And and so that's why again I think that
there's there's a lot of fraud that is
happening now of people who are having
AI do stuff and putting their name on it
and there's a little window of time when
you can get away with it. Uh but I think
that the real the best use of AI is
going to be one people using it to do
great things. And I talked about my my
late father's cancer. I mean that was
AI. I don't have the ability to an
analyze a whole human genome on my own.
Nobody does. So we would expect doctors
to be using and healthcare providers to
be using AI tools to optimize outcomes.
And so I think that's it's great, but we
need to define what it is and what do we
want? What's the role for humans and
whether it's human health care providers
or artists and other creatives or or or
really anything. And then we need
radical transparency. And so that was
for me one of the reasons why I insisted
having GBT5 as my named co-author is I'm
a I wanted to be a pioneer in this kind
of radical transparency.
>> Of your 10 AI commandments, do you have
a favorite?
>> Oh, you know, I think probably the last
one to live with awe, gratitude, and
love because we are so small. uh the AI
ten commandments. It's kind of like the
Aremis 2 astronauts looking back on
Earth and when they look back they just
see this little ball of life in this
dark empty mostly
lifeless universe even if there is and I
think there probably is life someplace
else. And so there we should have like a
level of awe of of just where we are.
And and and I think that when we do that
can kind of awe and love and and respect
animate so many of the uh of the other
things. And so um but you I to tell you
the truth I kind of love them all. And
every person who I have read side by
side the as written biblical ten
commandments and the AI ten
commandments, I have yet to meet a
single person who says at least when
read literally the biblical ones make
more sense than the AI ones. and and
then to say, well, given that these AI
ten commandments are drawn on from all
of our traditions, it's kind of a a
wonderful message that we're all part of
this this same thing, not just the same
humanity, but this same experiment of
life in in, you know, 3.8 billion years
on this planet that connects us to each
other and and all of life. And I think
that that perspective um is something
that's really beautiful and inspiring.
>> I love it. A week or so ago, Pope Leo
released his fourth encyclical which is
Magnifica Humanitus. Uh as I understood
it, the premise was technology is never
moral. It was in large part about AI. As
I read your book, it was kind of making
the opposite case of that. So, if you
could sit down with Pope Leo, what's the
one question you would want to ask him
or the thing that you would want to tell
him about?
>> Yeah,
>> it's great. And I I actually did a long
blog post on the papal papal encyclical
um which maybe you can can link to in
the uh in the show notes. And I have
spoken at the Vatican. Uh, and as a
matter of fact, I had a I was supposed
to meet with Pope Francis, but I had to
fly home uh to my original home in
Kansas City. Now I live in New York
because my uh my then 83year-old
mother was having her bach mitzvah. And
so I had to choose it was between my
mother and Pope Francis. And I I chose
my mother. But if I would have the honor
of sitting down with Pope Leo, what I
would say is I really loved the
encyclical. I really loved this
aspiration which I fully share to say
that these technological innovations
must serve humanity. They must serve
human values. And that's not going to
happen on its own. It's going to happen
with a level of consciousness, with a
level of aspiration. uh because no
technology comes with its own built-in
value system. It's up to us to infuse
our best values including our most
ancient and cherished values into the
technologies that we are creating. But I
had a couple of critiques uh of that in
the in spite of my extreme praise and
that was I felt like there wasn't that
much in the encyclical about the
incredible positive story of AI and and
related technologies about what it can
do to in health care and making our
agriculture more uh more efficient and
productive so we can feed more people
better uh with fewer inputs of land and
energy and and fertilizer and other
things and not to mention uh the
terrible risk and and the terrible cost
in suffering and pandemic threat and
antibiotic resistance of industrial
animal agriculture. So that was my my
biggest thing was to say let's it's
great to focus on the dangers and the
values and I certainly agree with both
but it felt a little bit unbalanced that
it didn't say how exciting these are and
that it is in my view and and I think
even in the church's view it's it's all
of our responsibility like if we had
health care providers who were slowing
down in treating cancer
because AI happened to be a technology
that was facilitating cancer care. I
would see that as a harm. At the same
time, you mentioned in your introduction
of me that I was a member of the World
Health Organization expert advisory
committee on human genome editing. As a
matter of fact, I was invited to the
Vatican to talk about that and was
invited to to uh write a a paper for the
Pontipical Council of Culture which they
which they published. And so that human
genome editing was made possible through
these revolutionary tools. And although
it is my strong view that we will have
genome edited human embryos that will
become humans in the future and we will
want to have it and it will be highly
moral and ethical to do it. Particularly
if we can change a 2-year life of
somebody born with a terrible deadly
genetic disorder into a 90 or 100year
fully realized life of somebody who the
same person who just doesn't have that
that single harmful harmful mutation.
But our conclusion of our committee was
we're not ready for it now. And so that
what happened in China in 2018 and 2019
in my view was Nermberg style human
experimentation and I'm against that.
And so that's why I think there's this
danger to feel like the question being
asked is technology yes or no. And I
think that's really the wrong question.
The the question is technology how best?
And we have different perspectives. The
church is coming from one perspective.
I'm not a Catholic. I'm a I'm like many
others in in New York City. I'm a Jewish
atheist with Buddhist leanings. And so,
you know, I I you know, do I think that
the very traditional dictates of
Catholicism
um are are going to be our our core
guiding principles? No, I I do not. But
do I think that there are values not
just of the Catholic tradition but of
all traditions that can contribute to
letting our ethics guide our journey in
into this very radical future? I think
it's it's very important and that many
traditions have that and many like the
Catholic Church. I thought it was great
that the pope um used this opportunity
to sneak in an apology for the Catholic
Church condoning slavery and
colonialism. It was just a paragraph. It
didn't have to be there, but it was
clear that Pope Leo wanted to find a way
to say that. And so I thought that was
that was very positive. But I think that
it's the important thing is balancing
it. It's not if a mean this is rational
optimism. It's not blind optimism. Blind
optimism can actually be quite dangerous
and blind pessimism, overwhelming
pessimism in my in my view can be quite
dangerous. But rational optimism,
measured optimism, thoughtful optimism,
I think that's what's required. And I do
I do think actually that this encyclical
was very very different uh from the the
um Catholic Church treatment of
Capernacus or Galileo or frankly even of
stem cell research which I think the the
Catholic Church is is getting wrong and
I've been very public including at the
Vatican saying that. Um, but I think
it's very positive and I hope that it
inspires people from other traditions
and other backgrounds and other
organizations to start putting out their
principles. I thought that the the
anthropic AI constitution was helpful.
None of these things single-handedly are
going to be transformative, but
collectively I think they can be very uh
constructive and and additive. You have
a PhD in history. You also mentioned
there's the seed of a panic around AI
right now. There's a certain backlash
against the building of data centers,
what AI will do uh to jobs, all that
stuff. As you think back through your
knowledge of history, do you have a a
favorite or a a panic from throughout
history that most reminds you of what's
happening with AI right now? You know,
there is always a panic at every
transition. I'm sure that there was a
panic a million years ago when people
started to control fire. I'm sure there
was a panic uh 12,000 years ago with
agriculture when life lives and and
lifestyles
started to change. Obviously, we're
familiar with the with the lites uh with
with industrialization. I don't know if
I have a favorite, but I do think that
all of these fears are actually founded
that, you know, the people who were
worried about giving up their hunter
gathering nomadic lifestyles. I think
that was a legitimate concern actually
in for many thousands of years in
agriculture. the people who were doing
agriculture were were less healthy in
many many ways than their hunter
gatherer nomadic uh nomadic
counterparts. There were a lot of people
who were harmed by industrialization.
But when we look back actually
industrialization was really great. It's
not coincidental that industrialization
rose and slavery ended at roughly the
same time because humans we had a lot of
stuff that we wanted done and it
required power and whether that was
human power or animal power or machine
power we still needed the power. So all
of these these changes come with with
upsides and downsides. you you mentioned
in both super convergence and your new
book the AI ten commandments how I how
AI can help us do all these amazing
things the AI opportunities if you will
is there one specific breakthrough in
healthcare in medicine in education that
makes you really stand up and say wow
that's going to it's going to change the
world over the next 5 to 10 years yes so
in every one of those areas but probably
the one that I'm most excited about is
education. Uh you and I, Stephen, have
great educations. We come from places
where there's a a lot of opportunity,
but there's a whole lot of people who
don't have great educations, who are in
places where there isn't a lot of of
opportunity. And that doesn't just suck
for them. It sucks for us because we are
beneficiaries of this cultural
inheritance, this scientific
inheritance, which is an inheritance of
humanity. You and I are speaking now in
English a language which we didn't
invent but we have this wonderful
language and has all these words that
help us form our thoughts into these
sharable forms. And so if and young
people around the world now through
smartphones which many many people have
uh can access AI tutors along hopefully
alongside wise human tutors and wise
human elders who are helping guide and
and instill values and morals and and
ethics. I think we're going to unlock
such amazing human potential that is
going to benefit everybody and that so I
just couldn't be more excited. And so
for me the most exciting applications of
all of these technologies are about
helping the world's most vulnerable
people. And so yes, I'm excited about
the future of oncology. And yes, we're
already making incredible innovations
now with pancreatic cancer for example
that people thought well that was a long
a long way off. Alphafold is now in
pretty much every country in the world.
At last record there were something like
two million labs had used it to advance
their research. And that's incredibly
exciting. But what I'm most excited
about is all of these people in all of
these poor places and slums and and
remote poor agricultural communities.
How many Einsteins and Mozarts and
Beethovens are in those places? And so I
think to unlock that that human
potential
>> is just an unbelievable gift not just to
them but to all of us. I think these
technologies if we work in that
direction can can help make that happen.
>> Jimmy, this has been a truly fascinating
conversation. What I'd like to do is
close the podcast with a quick lightning
round, a game of overrated and
underrated. So, what I'm going to do is
I'm going to name 10 things that kind of
fall within your wheelhouse and you can
respond to each one of them. Overrated
or underrated. Great.
>> What for you?
>> Uh, let's let's roll them.
>> Okay. First one, AI agents
underrated. It's going to be such an
exciting thing. We're just at the
beginning of what AI systems can do and
people are talking about making hotel
reservations, but there's so much that
can happen in scientific research and
and so many other areas. There will be
downsides. Bad things will happen along
the way, but I think they're underrated
because I'm really excited about them.
>> China's AI capabilities.
>> I think it's underrated. I mean, China
is really focused. Uh they've put a lot
of energy into AI. They have a really
smart smart people.
They are not yet. China isn't yet at the
level uh certainly of of the United
States, but it's catching up pretty
quickly and it's focused on on catching
up. So, I think still underrated.
>> In the same vein, open- source models.
>> I think underrated. I mean, that's the
big gamble. China is using the same
strategy that Google used with Android
to break the Apple monopoly. So they
thought, well, we're not going to win
the foundation model race, but we're
going to use an open- source model. And
frankly, they they were beneficiaries
when Meta was also losing AI. They said,
"We're going to go open source." So the
second place person always goes open
source. Uh and there's a a lot of
opportunity. I I still believe in
foundation models. I think, you know,
our great companies are doing amazing
things. Uh and I'm not saying that the
future belongs to to open source. You
know, people, you know, Apple did pretty
well by creating a walled garden uh
ecosystem. Um but I think there's a a
long runway for open source
>> human genetic enhancement.
I think it again I this is this optimism
I think it's underrated. No, I think
it's underrated in the sense you people
believe now that we're capable. People's
like oh you should engineer your your
kids to play chess or it's really
complicated. We are at the very very
early stages of this. We understand such
a tiny fraction of the full complexity
of human biology. And so um but this is
a a fundamental technology. We're going
to use it more and more in animal
agriculture and plant agriculture and
that's already happening. Multipplex
edits will allow us to do more changes
at once. Um, but our ignorance, our
relative ignorance of what of genetics
make it so that if we're doing a lot of
changes, even if we correctly identify
that making these changes in a human
will lead toward these outcomes, we
don't know nearly enough about what the
potential downside risks because it's
not like one gene is doing one thing.
Uh, our bodies are very very complex and
interactive and we we we don't know what
we don't know, but we know that it's a
lot. So I think that people are
overestimating what's possible now. Uh
but at this rate of innovation, I think
exciting things will happen in in the
future. But I but I do think that when
when engineers and computer science
people talk about engineering biology,
they're thinking think thinking of it
like an engineering task. But those of
us who are enscconced in the biology
world know just how complicated biology
is and how how little we know. So, I
think it's it's going to be slower than
people anticipate, but I think
fundamentally transformative over time.
>> A Matt Ridley favorite of Amara's law on
that one. Maybe
>> Amara's law is you you overestimate the
short term and underestimate long term.
Yeah. Uh number five, humanoids.
Humanoid robots.
>> You know, I think um I seem to have just
the same answer. So, I think humanoid
robots are realistic. Humanoid robots
are a bit far away, farther away than
people seem uh people believe. But I
think we're going to have huge advances.
And my I mentioned my new novel Virtuosa
is all about this and and it's what it
feels like for humans when they
experience a humanoid robot that doesn't
look like a human, but it's a humanoid
robot that can do magical things with
classical music that is emotionally
resonant for humans. And I do think I do
believe that we are going to have AI
systems that can write their own code,
which is the case in my novel, that can
imagine goals and work toward creating
them. So I I I I think we're going to
have humanoid robots. They probably
aren't going to look like humans. I
mean, we I have an amazing uh robot in
my house. After dinner, I put my dishes
inside of this robot and add soap and
press a button. And so we we may have a
bunch of um of robots that are
structured for specific tasks, but we
live in a world that is organized around
our form. And so there there will be
cases where we want to have humanoid
robots. And I'll just say this as for
anybody who's uh who's listened all the
way in this in this podcast. Yeah. I I
think we're going to have sex robots. I
mean, we're going to have robots doing
things that humans want, and that's
going to be crazy in the beginning and
normalized over time, like IVF and kind
of everything else that we're scared of
and then becomes normal.
>> I'm going to end with two, Jamie. Yes.
First one, data centers in space,
>> you know, that I I don't I don't feel
like I know enough to fully make a bet.
I like the idea and there are good
reasons to put data centers in space. It
solves certainly one of the big problems
of cooling and we can get a lot of data
back and forth on on satellites. So in
principle I'm excited about it. Um but
it's one of those things where it's
going to be harder. I mean when you're
writing science fiction you can just
imagine it's there. But when you're
building the thing at scale and it needs
to be cost effective that that's
actually a big task. But I I think it it
could work and ought to work and maybe
not for everything but for certain tasks
and I think it it will be positive.
>> Last one which is very relevant to you
writing books in the age of AI so
important and so hard. I mean, I, as I
said before, being a human, doing core
human stuff is going to be more
important than ever before. And our
greatest artists who touch the core of
their humanity and our humanity will be
more important than ever before. And I
think societies are going to have to
really invest in making space for human
artists being humans. We also need to
make a space for human creators
collaborating with technologies in new
and innovative ways that will will blow
our mind. But we really the the human
artist, the human soul, human
imagination, human uh creativity is so
important in and of itself and for
building the best possible future and
for actually having technology that
serves human needs. Jamie Medsel, thank
you so much for taking the time to sit
down with us. I want to give you a
chance to tell everyone where they can
find out more about your book, where
they can buy the book, where they can
follow all your work.
>> Oh, thank you so much. Uh, so the
easiest way to get everything is through
my website, jammetzel.com.
jamm.com.
Don't put a second e in my last name.
People have been trying it for years and
it's wrong. Uh, and you can get the
book. I mean, there's there's links on
that site. You can get the books on on
really anywhere, Amazon, your um your
local bookstore. I have a newsletter, so
if you go to my site, you can sign up
for my free newsletter, and I'm happy to
share my thoughts, such as they are,
with with anyone who signs up.
>> Jamie, thanks so much for your time, and
I hope we can do it again soon.
>> Thank you so much, Stephen. It's really
been a pleasure.