[00:00] Welcome to Huberman Lab Essentials, where we revisit past episodes for the most potent and actionable science-based tools for mental health, physical health, and performance. I'm Andrew Huberman, and I'm a professor
[00:13] of neurobiology and ophthalmology at Stanford School of Medicine. And now, >> Hi, hi Andrew. >> Great to be here with you. Your main focus these days is the neurobiology of speech and language. So, for those that
[00:30] aren't familiar, could you please distinguish for us speech versus I think about language, I think about words and just talking. If I sit down to think about the words I want to say very much. I mean, I have to think about
[00:47] them a little bit, one would hope. But, I don't think about individual syllables unless I'm trying to, you know, you know, accent something, or it's a word that I have a particular difficulty saying, or I want to change
[01:00] the cadence, etc. So, what in the world is contained in these brain areas? What is represented um to me is is perhaps one of the most interesting questions, >> Sure. Let's get into this, uh Andrew, because this is one of the most exciting
[01:17] stuff that's happening right now is understanding how the brain processes these exact questions. And speech corresponds to the communication signal. It corresponds to me moving my mouth and my vocal tract to generate words. And
[01:33] you're hearing these as an auditory signal. Language is something much broader. So, it refers to what you're extracting from the words that I'm what I'm saying. There's another aspect of it that we call semantics. Do you
[01:49] understand the meaning of these words and the words are assembled in a grammatical form. So, those are all really critical
[02:01] parts of language. And speech is just one form of language. There's many other forms like sign language, uh reading. Those are all important modalities for reading. Our research really focuses on this area
[02:16] that we're calling speech. Again, the production of this audio signal, which you can't see, but your microphones are picking up. There are these vibrations in the air that are created by my vocal tract that are
[02:30] picked up by the microphone in the case of this recording, but also picked up by the sensors in your ear. The very tiny vibrations in your uh ear are picking that up and translating that into electrical activity. It's such a complex
[02:45] species is is this speaking, not, you know, the extreme feats of acrobatics or especially when uh one observes, you know, uh opera or um people who, you
[03:01] know, freestyle rappers, you know, and and of course it's not just the lips, >> And you've mentioned two other structures, pharynx and larynx are the main ones that they um can you tell us just just educate us at a at a
[03:14] superficial level what the pharynx and larynx do differentially cuz >> I'll talk primarily about the larynx here for a second, which is that if you
[03:26] breath. So, even before you get to the larynx, you got to start with the expiration. We fill up our lungs and then we push the air out. That's a normal part of breathing. What is really amazing about speech and language is
[03:41] physiologic thing at a larynx. And what the larynx does is that when you're exhaling, it brings the vocal folds together. Some people call them vocal cords. They're not really cords,
[03:56] they're really vocal folds. They're two pieces of tissue that come together and through the vocal folds when they're together, they vibrate at really high frequencies, like 100 to 200 hertz. And the reason why men and women generally
[04:11] have different voice qualities is it has to do with the size of the larynx and the shape of it. Okay, so in general, men have a a larger voice box or larynx and the vibrating frequency, the resonance frequency of the vocal folds
[04:27] when the air comes through them is about 100 hertz for men and about 200 for women. So, you take a breath in as the air is coming out, the vocal folds come together, and the air goes through. That creates the sound of the voice that we
[04:41] call voicing. It's not just your voice characteristic, it's the energy of your voice. It's coming from the larynx. here. It's a noise. And then it's the source of the voice. And then what happens is that energy,
[04:56] that sound goes up through the parts of the vocal tract, like the pharynx, into the oral cavity, which is your mouth and your tongue and your lips. And what those things are doing is that they're shaping
[05:10] this the air in particular ways that create consonants and vowels. That's what I mean by shaping the breath. It just starts with this exhalation. You generate the voice in the larynx and
[05:23] then everything above the larynx is moving around, just like the way my mouth is doing right now, to shape that air into particular patterns that you can hear as words. >> Immediately makes me
[05:37] wonder about more um primitive or non-learned vocalizations, like crying or laughter. Are those produced by the language areas or do they have their own
[05:49] unique neural structures? >> We call those vocalizations. A vocalization is basically where someone can create a sound like a cry or a moan, that kind of sound. And it also involves the exhalation of air. It also involves
[06:06] some phonation at the level of larynx where the vocal folds come together to create that audible sound. But it turns out that those are actually different areas. So people who have injuries in the speech and language areas often
[06:20] And it is a different part of the brain. I would say an area that even non-human primates have that can be specialized, you know, for vocalization.
[06:32] It's a different form of communication than than words, for example. >> Speaking of storage of an ability to speak, you are doing some amazing work and have achieved some um pretty incredible well-deserved recognition for
[06:47] your work in bringing language out of paralyzed people. Essentially allowing people who are locked into a paralyzed state or otherwise unable to articulate speech using brain-machine interface, essentially
[07:02] translating the neural activity of areas of the brain that would produce speech into hardware, artificial non-biological >> So there are a series of conditions. Um they include things like brainstem
[07:19] stroke. The brainstem is the part of the brain that connects the cerebrum, which is the top part that does our thinking and a lot of the motor control, speech, and the nerves that go out to the face and vocal tract. So if you have a stroke
[07:34] there, you could be thinking all the wild creative intelligent thoughts you have in the mind in the cerebrum, but you can't get them out into words, or So, that's a very severe form of paralysis called brain stem stroke.
[07:49] There's another kind of conditions that we call neurodegenerative where the nerve cells die basically or atrophy in a condition called uh ALS. That's a very severe form of paralysis. In its extreme form, people essentially lose all
[08:04] voluntary movement. The muscles to their diaphragm and their lungs essentially field, these are kind of like the most devastating things that can happen. This refers to this idea that you can have completely
[08:22] intact cognition and awareness, but have no way to express that. No voluntary movement, no ability to speak. And that is devastating because uh psychologically and socially, you know, you're completely isolated. That's
[08:37] what we call locked-in syndrome. And it's devastating. So, we've been studying this patterning of electrical activity for consonants and vowels. And essentially, once we figured out a lot
[08:49] of these codes for the individual phonetic elements, part of the lab For people who have these kind of paralysis, could we
[09:01] intercept those signals from the brain, the cerebral cortex, as someone is trying to say those words? And then can we intercept them and then have them taken out of the brain
[09:14] through wires to a computer that are going to interpret those signals and translate them into words. So, we started a clinical trial. It's called trial was a man who had been paralyzed for 15 years. He was in a car accident.
[09:32] He actually walked out of the hospital day after that car accident. But the where he had a very large stroke in the brainstem. And that turned out to be devastating. He didn't wake up from that stroke for
[09:48] And when he woke up from that coma, he realized that he couldn't speak or move his arms or legs. As he told me or communicated to us, that was absolutely devastating. He wanted really to die at that time.
[10:03] >> He could blink his eyes. He had some limited mouth movements, but couldn't produce any intelligible speech. It was like completely slurred and incomprehensible. He survived this injury. A lot of people who have that
[10:17] kind of stroke just don't survive. The way he actually communicates, because he he improvised and had his friends basically
[10:29] put a stick attached to his baseball cap. And because he could move his neck, he would essentially type out letters on a keyboard screen to get out words. In fact, this is how he communicated was through a device that he would
[10:43] essentially peck out letters one by one by moving his neck to control this stick attached to his baseball cap. He hadn't really spoken for about 15 years. trial. It was, you know, something that our hospital and also the FDA, you know,
[11:00] that we had done, there was some basis for for why this might work. And so, we did a surgery where we implanted electrodes
[11:13] onto these areas that control the vocal tract, the areas that control the larynx, the areas that control the lips and tongue and jaw movements when we normally speak. These are areas that presumably may be active. That was
[11:26] our hope. And he underwent a surgery, a brain surgery, where we put an electrode array, and we connected it to a port that was skull to screw to a skull. The port actually goes through his scalp, and he's lived with this now for last 3
[11:40] years. So, he has an electrode array that's implanted over the part of this brain that's important for speech. It's connected to a port. And then, we connect a wire to that port that translates those
[11:54] and converts them into digital signals. We put them through a machine learning or artificial intelligence algorithm that can pick up these very, very subtle
[12:07] uh in in the brain activity, and translate those into words. And this is something that took weeks to train the algorithm to
[12:19] interpret it correctly. But, what was incredible about it was to see how he reacted. He would be prompted to say a given word like, you know, outside, for to say it, and finally those words would appear on the screen. And what was
[12:36] really amazing about it was you could really tell that he like got a kick out of that because, you know, his body would shake in a way, and his head would also realized that when he was giggling, it kind of screwed up the next
[12:51] word's decoding. >> Is that a bug you've since just to tell him to stop giggling. The way this worked was we trained uh
[13:04] this computer to recognize 50 words. We started with a very small vocabulary. That's expanding as we speak. I think that this is just a matter of time before these vocabularies become much, much larger.
[13:16] But, we started with a 50 set of words. We created essentially all the possible sentences that you could generate from those 50 words. Why that was important was you can use those all those possible sentences to create a computational
[13:30] model computer model of all the different word combinations to give different sentences given those 50 words and then you can essentially do what we call auto correct. It's the same kind of thing that we do when you're texting for
[13:44] know, because it's context what to correct it. So because the decoding is not 100% correct all the time. In fact, it's far from that.
[13:56] It's really helpful to have these other features like auto correct the stuff makes it correct and then updates it. So it's a combination of a lot of things. It's the AI that is translating those brain activity patterns, but it's also
[14:12] that you know, you put all together and then all of a sudden it starts to work. That was the first time that someone was paralyzed and could create words and sentences that was just decoded from the brain activity.
[14:27] right? Elon Musk's company. While brain machine interface of the sort that you do and that other laboratories do has been going on for a long time, there's
[14:39] been some press around Neuralink about the promise of what brain machine interface could do. What are your thoughts about manipulating neural circuitry to achieve supra human or super human or super physiological
[14:53] functions? And here we don't even have to think about Neuralink in particular. It's just but one example of companies and people and laboratories this. >> It's a really interesting time right
[15:06] now. The science has been going on for decades. The work that we've done in this field that you call brain machine interface has been going on for a while and a lot of the early work was just trying to restore things like arm
[15:19] movement or having people or monkeys control a computer cursor for example on the screen. That's been going on for decades. What's been really new is that industry is now involved and some some of this now becoming commercialized and
[15:33] we're starting to see us now cross over to this field where it's no longer just research that we're talking about medical products um that are designed to be you know surgically implanted in some cases. You
[15:47] require surgery. The specific question that you were asking about is an area that we call augmentation. So, can you build a device um that essentially
[16:01] enhances someone's ability beyond supernormal? Super memory. Super communication speeds beyond speech for example. Superior uh precision athletic abilities. I think
[16:17] that these are very serious kind of questions to be asking now because as you mentioned, the pathway so far is really to focus on these medical thought enough actually about what these kind of scenarios are going to look
[16:34] all the ethical implications of what this means for augmentation in particular. There's part of this that is not new at all. Humans throughout
[16:46] function. Coffee, nicotine, all kinds of medications that cross over from medical to consumer. That is everywhere. So, the pursuit of augmentation or performance
[17:02] or enhancement is really not a new thing. The questions really as they relate to neurotechnologies for example have to do with the invasive nature. For example, if these technologies require surgery
[17:16] for example to do something that is not for a medical application. Again, there that is not exactly new territory, either. People do that routinely for cosmetic kind of procedures, for physical
[17:30] I do think that provided the technology continues to emerge the way that it does, that it's going to be around the corner.
[17:43] think it's going to be like, can we easily memorize every fact in the world? But in forms that are going to be much more incremental, and maybe more subtle.
[17:55] In many ways, we already have that now. Like, for example, you don't have to access to all information in the world. You just have to have, you know, your through uh a brain interface, I definitely wouldn't rule that out. But
[18:13] think about this, that systems that we have already to speak and to communicate have evolved over, you know, thousands and millions of that have bandwidth of millions of neurons.
[18:30] There's no technology that exists right now that people are not even in research labs, that come anywhere close to what has been evolved for those natural purposes.
[18:44] So, I'm essentially saying two sides of this, which is we're already getting into this now. This is not new territory, this topic of That's part of what humans do in general. But we are entering this area
[19:02] of like enhanced cognition, um these areas that I think the technology is going to be the rate-limiting step in how far it can go. And we have not had the conversations about number one, is this
[19:15] what we actually want? Is this going to be good for society? Who gets access problems. >> Could you tell us what you're doing in terms of merging the brain-machine interface with extraction of speech
[19:28] signals from people who are locked in like Poncho with facial expressions? >> Sure, yeah. I'm here with you in person. We could have done this virtually, probably. It's pretty easy to do that. We could have recorded this
[19:41] expressions and to understand other forms of is nonverbal. The expressions that you're making. For example, if you have
[19:55] a quizzical look on your face if I'm saying something not clear, that's a sign to me that I need to rephrase it or to say it in a different way or slow down. Facial expressions actually are really important part of the way we
[20:08] speak. And there's two things. It's not just the expressions of like how you're feeling and perceiving what I'm saying, but it's also seeing my mouth move. And your eyes actually see my mouth move and my jaw
[20:21] move in a particular way that actually allows you to hear those sounds better. So, having both the visual information but also the sounds go into your brain is going to improve intelligibility, also make it more natural. And the
[20:35] reason why we're also very interested in this idea of not just having text on a screen, but essentially a fully computer animated face like an avatar of
[20:49] the person's speech movements and their facial expressions is going to be a more complete form of expression. Now, you can imagine right now that might just be someone looking at a computer screen interpreting these signals, but
[21:04] I think the way things are going in the next couple of years, a lot more of our social interactions, more than even now, are going to move into this digital for most consumers, but it also has really important
[21:19] implications for people who are disabled, right? And whether the how how are they going to participate in that? And so we were thinking really about for people like Poncho and other people who are paralyzed, what other forms of BCI
[21:34] can we do in order to help improve their ability to communicate? So one is essentially decode, you know, essentially their their expressions or
[21:47] the movements associated with their mouth and jaw when they actually speak to improve that communication. >> So do you envision a time not too long from now where instead of tweeting out something in text, my avatar will I'll
[22:00] I'll type it out, but my avatar will just say it. It'll be a an image of my >> That's what we're working on. That is going to happen and it's going to happen soon and there's a lot of progress in that and again, we're just trying to
[22:14] enrich um the the field of you know, of communication expression um to make it more normal. And we actually think that having that kind of avatar is a way of
[22:27] prosthetic. That's the device that we call it. It's a speech neural prosthetic. That is going to be the way that can help people learn how to do it the quickest, not necessarily like trying to say words and having it come
[22:40] on a screen, but actually have people embody, feel like it's part of that illustration or animation. >> I get a lot of questions about stutter.
[22:52] What can people stutter do if they'd like to relieve their stutter? >> Stutter is a condition where the words can't come out fluently. So talked about this distinction between language and speech. Stuttering is a
[23:09] problem of speech, right? So, the ideas, the meanings, the grammar, it's all there and people stutter, but they can't get the words out fluently. So, that's a speech condition and uh in particular, it's a
[23:22] condition that affects articulation, specifically about controlling the production of words in this really coordinated kind of movements that have to happen in the vocal tract to produce fluent speech.
[23:37] And um stuttering is a condition where people have a predisposition to it. So, there's an aspect of stuttering, you are a stutterer or you're not a stutterer, but
[23:49] people who stutter don't stutter all the time, either. So, you could be a stutterer who stutters at sometimes, but not others. stuttering and anxiety is that anxiety can provoke it and make it worse. That's
[24:08] certainly true. But, it's not necessarily caused by anxiety. Can essentially trigger it or make it worse, but it's not the cause of it per se. So, the cause of it is still really not
[24:22] clear, but it does have to do with these kind of brain functions that we've been in order to produce normal fluent speech, we're not even conscious of what
[24:36] We're not conscious and if we were, we would not be able to speak because it's too complex. It's too precise. It's something that we and we do it naturally, right? It's part of our programming and part of what we
[24:53] learn inherently and then, you know, it's just through exposure. So, stuttering is a is essentially a breakdown at certain times
[25:05] in that machinery be able to work in a really coordinated way. controlling the vocal tract. Let's say speech is like a symphony. In order for it to come out normally, you've got to have not just one part, the larynx,
[25:21] but the lips, the jaw. They can't be doing their own thing. They have to be very, very precisely activated and very, very precisely And so, in stuttering, there's a breakdown of that coordination.
[25:37] >> If somebody has a stutter, is it better to address that early in life when there's still neuroplasticity and is very robust? And if so, what's the typical route for treatment? I I have to imagine it's not brain surgery
[25:50] that that people can talk to and and and they can help them work out where >> Yeah, exactly. I mean, part of it is about that anxiety, but a lot of it
[26:05] really has to do with um therapy to sort of like work through and think of tricks basically sometimes to create conditions where you can actually get the words to come out. A lot of some
[26:19] started itself is is very hard. You want to start with initial vowel or consonant, but it won't emit. So, a lot of the
[26:31] create the conditions, you know, for that to happen? There's another aspect to it that I find very interesting is that um the feedback, essentially, what
[26:43] we hear ourselves say, for example. Every time that I say a word, I'm also hearing what I'm saying. So, that's what we call auditory feedback. That turns out to be very important and sometimes when you change that, it can
[26:56] actually change the amount someone stutters for better or for worse. And it it's giving us a clue that the brain is not just focused on sending the commands out, but it's also possibly interacting with the part that
[27:10] is hearing the sounds and there's something that might be going on in that So, there are individuals that are stutterers, but they don't stutter all
[27:22] happening in those particular moments where this very, very precise out fluently. >> And you have to say from the first time
[27:35] we became friends uh 38 years ago. >> Something like that. >> To be sitting here with you today for me is a absolute thrill, not just because we've been friends for that long or that
[27:47] science, but because I really do see what you're doing as really representing that front absolute cutting edge of of exploration and application. I mean, the
[28:01] story of Poncho is but one of your many patients that um has derived tremendous benefit from your work and and now as a chair of a department, you of course cord, etc. So, now on behalf of myself and and
[28:17] joining us today to share this information, but also just for the work you do. It's truly spectacular. So, thank you ever so much. >> Thanks.