[00:02] heard about a robot that was going around handing out popcorn to people. >> Yeah. >> Too much pressure on the knee joints, I >> Like humans. >> And yeah, and I was thinking like [00:17] >> [laughter] >> The less friction the better. goal is to get more work done. >> If we had another 1 million X in the next 10 years, what do you think could we do that we cannot do now? [00:33] >> All right, two of the hardest questions. >> Good to see you, Spencer. >> Good to see you, Karoly. >> Uh so today we announced a few things. Uh first we've got Isaac Lab with Newton [00:48] updates. And one of the the best things about this update is that not only is it going to help with uh developers really being able to scale out, but it also enables them to uh fully take advantage of Newton. This is one of those things [01:01] we've we've really been looking forward to. Um Newton as a physics framework came out uh last year, as you know. And integrating it into Isaac Lab has And the reason is because we're moving off of PhysX and and over to Newton. So, [01:16] uh these improvements are really going to bring some developers quality of life excited for that. The second is we have some DGX Station blueprints. And these are going to give people a really good place to start for um both training up [01:30] being able to leverage these DGX Stations, which are just fantastic if you have one, I I think you'll have a And the third is Agile One SN. This is a reference platform that we built [01:43] alongside with Agile. So we we worked with their fantastic Agile One platform. Uh and we've worked to ensure that with has everything from Thor integration all the way to optimizing all the the firmware and all the software stack on [01:57] the box, you already have Groot. It's ready to go for developers to get to the hardest things that we find with development on on like the hands. So, with this one, it ships with the Uji hands. [02:13] Bringing the hands onto the robot, as well as just getting the software stack up, can take months and, you know, weeks depending on how how familiar you are with these platforms. And so, just providing it with [02:25] something in that we're proud to ship, and something that we use ourselves, um getting going on on their actual workloads. We don't want them to have to fuss around with something that we can just solve for them if we can. [02:38] >> Less friction, the better. Yeah. The the goal is to get more work done. >> So, we have had LLMs for years and years now, but physical AI robots, Why is physical AI so much harder than LLMs? [02:53] >> Well, let's step back for a moment. LLMs had the advantage that humans have been writing down our history for as long as as we could come up with different modalities, whether it's writing or taking photos, [03:05] or just recording just like this. And so, having natural language and just had the basis for it. We already had a ton of content. So, training up these models really became an exercise of how do you take all of this unstructured [03:18] data, and how do you start extracting features, which then become feature these models to be more and more proficient as they go along. And then, we started adding different modalities to it, vision and audio and things like [03:30] that. But, for physical AI, you don't have any data set that already encapsulates all of the physics of the world, how you're interacting with the world. We talk about how you might act in the world, and so, the actions, and [03:43] know. We're very good at having VLMs be able to describe how people are how people, objects, agents are actually they interact with the world, you don't have that physical data, and that's [03:55] the critical part. And the reason is because when you're working with things manipulate the world and the world in turn manipulates back, right? In coupling. If you have bidirectional coupling, when I make some when I make a [04:11] might push back, right? It's going to be if I have a soft object on a rigid body, soft on my my hand, which is a soft object on this rigid body. Um but that's really difficult. So, one, you have to have the physics solvers [04:27] emulate the reality [clears throat] the the physics of reality in such a way that we can then start applying it into the training of the these robotic policies. On the other side, you still have to have all [04:40] of this data that's capturing what these dynamics are really supposed to be like. manipulation interaction data, if it's going to be motion data, even trajectories of, you know, we watch how robots today we have to do imitation [04:54] learning to start because it gives us somewhere it gives us a set of actions train robot policies that are more robust after taking imitation, you know, we'll do is we'll take teleop. You take a human and they're going to operate a [05:11] robot, they can capture all the data on top of that embodiment. Now, seeing improvements where you don't need to you don't necessarily have to only teleop. Teleop still has the same limitation that we have with humans. [05:26] Anyone that thinks that teleop is easy has is obviously never teleoperated It's expect because there is no force feedback for a lot of these. And because numbing your hands. >> That's how you the brain is wired that [05:40] you stop when you feel something. >> it stop when you feel sometimes you're and you have the robot here and the person's here and so, you actually have to kind of project in your head what the robot is seeing as you're doing this [05:53] from the side and that's just wildly difficult. So, trying to assemble huge undertaking. And so, what we've seen over the over the last year is you can finally start taking human data, right? You have egocentric data. I'm [06:07] just going to watch from a camera what I'm doing with my hands, and that gives take some data of the human body from the side, and I'm going to extract from interaction. And from there, I can start getting data [06:20] for how I can train these policies. Um so, it's it's really um it's really a challenge for us to not only define what is good data along with how to capture the data. And then on top of that is how best to use the data. And then really at [06:34] problem. I only have a limited amount of real data because it takes me so long x amount of money to get the environment. able to use simulation. And simulated data, while you can get vast amounts of [06:49] extremely high-quality compared to real data because we're still trying to extraction, right? How are How do you identify what features of simulated data are really going to impact the model performance when you when you [07:04] pair it with real data. And so, there is this, you know, law of large numbers for simulated data. And so, co-training them together, data together as you're training these policies, um becomes a bit of an art. [07:18] hard is because when you put all these problems together, and then you need to multiply it by n number of environments. We're not just trying to simulate this train a robot or an agent to be able to operate in [07:31] we're talking about terrestrial environments. Warehouses, you know, houses, data centers, even on the road for data centers, even on the road for autonomous vehicles, in the sky for for [07:44] you know, aerial like they have these wonderful delivery drones, you know. your lunch and it just comes to you in a park. But then we have to start going humans. It's just dangerous. We're going to go places deep down in the ocean. You [07:59] need to go places deep into the earth. And all of these are are, you know, extremely data sparse. Because how much data do we really have for all of that? yeah, a multi-dimensional problem and a lot of it is is we're trying to solve it [08:12] can and then how do we find the best ways to simulate that reality as best as we can. >> That was awesome. >> That was awesome. I've heard about so much like like it it sounds to me like [08:26] there is some sort of maybe an ideal data diet for for training physical AI. Like like LLMs you we have the textual data and we have the the multimodal [08:38] image this and that. But for this kind of thing, you have [clears throat] you have traditional handcrafted simulators really close to my heart. And you also have the neural world like [08:55] So, how do you build the ideal diet out of good at? What what are the good proportions? because I I I bet that if [09:07] you don't have the right proportion in the training data, then then you get something that doesn't generalize well. So, how that sounds that? >> Yeah, that's that's a really wonderful [09:19] >> [laughter] >> It is a if anything the nuance of of data mixture is really is kind of the right now I would say it's it's the art [09:32] in and of itself. So, let's take let's just kind of build our way up to up to take advantage of is base VLMs. Right, we already have world models that essentially are trained off of [09:48] a lot of world data that it's already seen. Right, it already knows what it's you start working in with the neural simulators, basically, how do we take model and then use the world model to simulate reality. What you start finding [10:02] like physics. And it hallucinates things like physics because we've trained it on on a ton of data um is not necessarily physically these learned physics. But, what you really want is to be able to ground it [10:16] uh with high fidelity physics for the types of things that matter. And a lot you know, the manipulation bits. If you're trying to walk through the world, consistency. How do I make sure that as I walk through a world, a simulated [10:30] world, that it retains coherency, so that way it still retain you know, I world is still the same. And the other And the other hand, when you start to manipulate the world, the physics matter. So, if you imagine [10:44] holding a ball and you know, last year, I think the the models have gotten just remember these orange balls, you pick one up and they drop it and just And it it hallucinates not only the physics, but it has hard with, you know, [10:58] But, what we can do now is we can take these world models that we're now leveraging as neural simulators that give you great diversity and give you great variety for all the types of observations that you [11:11] multimodal. So, can they Can they output actions? Can they output touch? Right? How do you output tactile feedback, force feedback, acoustics, all these types of things? On the other hand, I can also use conventional [11:25] grounding. Because with conventional simulation, >> It doesn't hallucinate, but it's extremely tedious to build an environment. It's extremely hard to create diversity because it relies on on [11:37] with the with the with the the recent advancements in in these especially things like Astra and and Opus 5.5, what we're starting to see is [11:51] a lot of the challenges, the bottlenecks that we had before where a human would scenes has started to become, you know, relatively negligible. But, it What it means is now we can have a lot of conventional simulated data that we can [12:07] simulations. So, you can do a few things. You you use neural simulation for things like automatic domain randomization. So, one one way to think of this is as you're training a policy, a robot may be [12:20] a visual sensor. And this visual sensor is perceiving the world, but it's just because all you care about really is the physics. So, you're trying to randomize across an envelope of physics. That's really the goal for for you know, [12:34] manipulative robots. You're not trying to get exactly the table. What you're trying to do is approximate across um, uh, uh, across an envelope, you know, what are the the typical distribution of physical [12:48] properties of this? That way as it trains within this, you know, a few iterations, and you know, we have all these different clones of this one robot. It's basically learning across what every type of table may be like, [13:01] but visually it may not be very realistic because all you're worried main bottleneck that we run into is rendering pixels. But what if you could take that simple rendering, that simple observation, and [13:15] you pass it over to a world model that then starts domain randomizing across things like materials and lighting and deformations and environment types, and then feed that back into the policy? So, the policy is actually learning both the [13:27] conventional physics, but it's getting the observations coming from a neural of ways that we can, you know, create these uh, hybridized training environments. And it's just the start of it. So, we're we're pretty excited about [13:40] this. Um, but we do need a leverage conventional and neural simulation together. And a way to think of it is they're they're siblings. So, a neural simulator, these world models, they're only as good as the data you give it. [13:52] for it because of all this this physical data that we just talked about. So, you don't have all this physics grounded data. And the better we get at um, at generating physics data in conventional simulation and capturing of [14:05] building up the data sets that these neural simulators, know, they're able to use uh, they're able to simulate the world much more But, it's going to be dynamic by dynamic. So, take a subset of the [14:21] whether it's, uh, you know, we'll have linear deformables, things like ethernet cables are not the same as threads on on your shirt, they're not the same as spaghetti, which, you know, spaghetti is not the same as some other things that [14:34] So, trying to figure out how do you actually model each of these types of dynamics, and that's just one type of dynamic. You have cloth dynamics, you've got, um, you know, uh, things like particle physics, [14:47] need to to model to get to the whole real world. You're trying to get the entirety, um, of the real world. And so, it's going to be a bit of, uh, conventional simulation and real world data, and they reinforce these neural [15:00] get to learn the physics based off a grounded physics. Now, they're doing better at simulation, and then we can start moving on to different, you know, a different subset of of, uh, dynamics to model. So, um, it'll be step-by-step, [15:12] and these agents are helping us make even further progress much quicker. So, >> Yeah. No, that answered 10 other questions, too. >> But, I I really love that, because essentially what is what is happening is [15:27] that is that you imagine a robot that doesn't just learn in this world or a simulated world, but 10,000 other worlds, too. Like, what if we were on know, what if hamburgers flew everywhere? That that would be great. [15:43] >> And and and neural networks like to memorize things if if you let them. And if you don't have enough training data, it it just wants to memorize. Yeah, yeah, and and over fitting and all all kinds of terrible things happen. But, [15:57] but if if you have this kind of training data, you cannot memorize 10,000 understand. And and that's the only way that that it works. So, I I love that. I think that's that's really cool. [16:11] Now, there is this, uh, paper from the robotics lab called Sonic. And you you can upload it into a robot, and it will be able to walk and and do useful things without falling even crawling and dancing works for different [16:24] crawling and dancing works for different kinds of body morphologies. That is insane and all this in 42 million parameters. That's like a grain of sand today like it's so small and it makes decisions in [16:39] real time. Also open weights and free for all of us forever. So so first of all thank you for that. That's that's awesome. And the other thing is how do you cram so much intelligence into such a small [16:53] go back to the subset analogy right if you're trying to model a subset of things what Sonic is really working on is whole body control. It's not working on every part of the system right so you can actually [17:08] have these smaller models that are able to have such great such great diversity and and capabilities. Now the challenge with Sonic and we're always making much more [17:22] acknowledge that the challenge is that it's extremely data hungry. You need to have a lot of demonstration data for Sonic to work really well. But Sonic once it works very well you can apply it along with you know Groot [17:35] or another another VLA of your choice to do the high level reasoning and that gives you the ability to have whole body control along with across tasks and motions and things like that. I think the the most important [17:50] thing for us to to remember is Sonic is that first step right it basically proves that you can have whole body tracking and you're taking all of this data of how humans move and then compressing it down into this model that [18:04] allows you to to give you whole body control. Now for those that aren't super familiar with whole body control one way to think of this is when you're told to pick up a box a heavy box we're always told to lift with [18:16] right don't bend with your hips. The the thing that you're going to do is you actually have to articulate your entire body in order to reach that lot of manipulative tasks, which is why whole body control is important. When [18:30] VLAs first started, it was actually separating the lower body and the whole body or the the the upper body. So, basically, it couldn't walk and drink Right? And the reason that this isn't very uh this isn't very useful is [18:43] because of the fact that you have to actually move up to a static position But the moment that you need to start reaching across, you're not able to body control maneuver. And so, that's actually what what these are trying to [18:56] inherent lower uh lower-level uh lower-level models. zero. Uh and then system one and system two are the VLA. And And of course, as we move on to world action models, um the nomenclature might change. [19:11] >> Mhm. I I got my 10,000th reason why physical >> Everything's just a little difficult. >> need you need system zero. So, you need like incredibly quick, rapid decision-making because like, do I take [19:26] a step or how do I stay upright or something? You cannot just give it to a back in an hour." >> Exactly, yeah. You need to have >> And the body um there's two types of [19:39] environment, right? The first is you can you can react to the environment, which you're just moving. So, if someone throws a ball, you you very quickly grab it. The second is you're you're reasoning about what you're trying to [19:53] Right? So, I'm looking at something, a challenge. It's a long horizon task. And So, how am I actually going to be leveraging the body? So, whole body control gives you both. It allows you to have reactions and it also gives you the [20:06] ability to leverage the the whole body as part of the the reasoning and and uh >> Mhm. >> In an interview about a year ago, you talked about neural simulators with world models coming online soon. [20:22] >> Sure. >> Accountability. Um Oh, actually, this that's a a wonderful because I'm super excited about this. Neural simulation, we're starting to use [20:37] Cosmos for simulation. Just like I mentioned, where you're blending conventional simulator, like Omniverse. And so, Isaac Sim, Isaac Lab built on Um and Newton, which would have all of the uh it's a framework for allowing you [20:51] to bring in different solvers and and different AI physics, you know, neural surrogates of of different um dynamics that you're trying to model. got Cosmos, which is that world model that we can then use for simulation. And [21:04] we're starting to see it being applied for a few in a few ways. The first way is just like I mentioned before, you can use it for domain randomization. You can that's the first first path. The second path is you can use it as a simulator [21:18] itself. So, if I train a policy, I could use this neural simulator to actually evaluate that policy as well. The limitation of this is, of course, just like I mentioned, if the if the simulator itself is not able to properly [21:32] simulate the the physics dynamics that you're trying to model against, then, policy is, you know, you you're not really quite sure if it's if it's going to succeed in in real. And so, that's where we have to blend neural simulation [21:45] and and conventional simulation. And as the neural simulators get better, uh and more and evaluate more and more policies. And then, the third one is models are becoming the policy themselves, and that's where you get [21:57] world action models. This is where we can leverage Cosmos as a backbone um for the the next variants of group. >> There is something incredible there. So, you said that essentially it can in a simulation run the verification. It can [22:12] assign a score and and and train maybe a new policy. >> You can imagine it like um You'll have to have a few, right? The first is you have a policy. And then, the policy is enacting inside of this [22:26] environment. So, we can use a cosmos to be this environment. So, you have Groot that's doing something inside of a cosmos world and then you're about trying to make some distinction as to whether or not it's successfully [22:39] you're testing under evaluation. Um you have a reasoning model another another VLN. This could also be cosmos but it's actually watching these and evaluate the metrics or give quantitative metrics based on what it's [22:54] quantitative metrics based on what it's observing within that simulator. And so, right now we're we're using Isaac Lab for this and we're we'll eventually bring in cosmos as back end as well. But you can think of we call it sensitivity [23:07] take a policy, you put it inside of Isaac Lab and Isaac Lab has a ton of environments. These environments have scenarios and within those scenarios you have defined metrics or criteria of success. And as the as the policy is [23:21] enacting inside of the simulator, you're basically getting these metrics out know, where is it failing and that's extremely important. Based on the task, grab the bottle isn't enough. You decompose that into multiple steps that [23:36] way you can identify where in that task is it starting to falter and where is it you know, performing strongly or where is it not. And then within those subtasks you're trying to identify based on a bunch of criteria what are [23:49] Is it the things like collision or things like you know, is it lighting it that that's actually impacting these policies? And so, this becomes a sensitivity analysis. From there, you can start working on failure-based [24:01] learning which is just how humans learn. The more you fail and analyze why you failed, the more you can actually retrain yourself to likely succeed the next time. And so, we can use cosmos as a as a back end a simulator for this as [24:16] And you have your policy going in and then we can evaluate it. This is is very using these neural simulators for it and we're doing it for conventional you just step forward how this how this plays out. Having a hybrid simulation [24:32] then you have physics grounding da da da da but then you're able to start evaluating is this policy going to work in real in But it also gives us a little bit extra. As you start evaluating the policy [24:47] deploy them and you see what you know how they're actually performing, you can start evaluating the simulator itself too. This is why we actually see simulation used for evaluation than it is for core training across a ton of [25:02] twofold. The first is conventional simulators like we mentioned are extremely hard to build and so you can only simulate accurately a certain subset of dynamics and so right now is mostly rigid bodies and so [25:14] we're we're getting now into rigid soft bodies but when people started they reason for that is because it's too hard to do that in simulation. Um originally. >> Me [laughter] neither. >> But these these agents are getting [25:28] pretty good at it and and now we have really wonderful solvers that are dynamics. So now you have it in simulation and the trust your simulator which is that sim to real gap. And that's really what that [25:42] next step is. You have the ability to have a neural simulator and a conventional simulator as your world and you're trying to test a policy across different environments, across a a bunch of different physics dynamics. And then [25:54] have to ask ourselves is well if you're in simulation um how do I know that it's going to transfer to real or how confident am I I think that becomes >> incredible because I I visit robotics [26:07] >> incredible because I I visit robotics labs around the world and almost every single time that's what I hear that that in the simulation things work great and world they are like in despair. Like nothing works and and you're you're [26:21] looking to close exactly that gap. Like that's the hardest part, right? >> [laughter] >> You know because no matter no matter what you can always tune something to have it succeed in simulation. And just [26:34] because you you succeed in simulation, there's no guarantee that it's going to you know just transfer zero-shot to the the platform that you're targeting. Now, that's when you're using a generalized [26:47] simulator, but as you watch people, they take our they take our platform, and they'll make it extremely explicit for the types of platforms that they're them the best thing to do is if you build the [26:59] going top to bottom, you're actually understanding all of the different tunings and different, you know, let's say it's the actual torque envelopes, and you've got all these different parameters that we may not get [27:12] just off of a spec sheet. And so they're they're able to start testing, okay, I I these things, I go and transfer it to the real robot, I see, you know, this type of difference. Um how am I actually able to start tuning back and forth? And [27:26] start closing that sim-to-real gap. But the real question is, how can we do it in such a way where you could bring a new embodiment very quickly uh and train a policy across many embodiments, right? My My hope would be I would love to [27:40] train a skill across many embodiments at a time, not just have one embodiment And if you're able to do that, that generalized simulator then has to cross that sim-to-real gap quite quickly. And so you have to be able to identify or at [27:53] um how accurate is my simulator, how much does it represent reality and and the the realistic system that I'm going to be putting this policy onto. Um and but it's that next challenge because now we finally have these physics solvers [28:08] that are able to simulate the types of dynamics that we need, right? A lot of warehousing, industrial environments, data centers. Um these are known areas, and so you have known known set of physics that you [28:21] from there you can start working on that sim-to-real gap and and narrowly closing it. And sim-to-real Um you didn't ask this, but I'll I'll >> Thank you. >> The sim-to-real gap is is actually not [28:35] one phase. It's a mix of The reason why we do teleop data is because if you can teleop on the robot that you're going to deploy on, then the data you capture is already kinematically mapped to the robot that that's going. So, you don't [28:49] learning. You don't have to retarget any of that data. Um the the difficulty you have with human data is you have to take a human embodiment and our embodiment, while it looks like, you know, these robots may look human, um the joints are [29:03] place. They may not have the same amount of joints. And so, kinematically, it's it's different. So, you have to retarget those to this new embodiment. Um so, Sim-to-Real starts with the data collection, goes all the way through [29:15] data generation, how you train it, how you, you know, train and configure uh policy, and then, of course, as you're evaluating. Are you evaluating appropriately for um for the world that you're you're going to be deploying it [29:27] on and and and the platform. And then, of course, how you deploy it and how you loop. And so, Sim-to-Real is often thought of as this one stage, but it's >> It's really interesting in the in the Google DeepMind Lab, I remember uh [29:42] seeing these these robots that did football, soccer, and they had a problem because they got injured over time. Like like this knee joint that there was too much pressure on the knee joints when kicking the [29:55] >> And yeah, and I was thinking >> 16 anymore? >> [laughter] >> And they actually had to be explicitly trained to, you know, not stand on one leg for too long and just ways to, you [30:10] pressure a bit. It's it's just like humans. Is that something that you can >> So, so I'm glad you you mentioned that. It's actually something that when you enough training data and you've actually simulated across enough, right? Because [30:26] it's one of those envelopes. So, an envelope um the the distribution of uh of actuation, right? Um perfect wrist motion versus imperfect wrist motion or [30:38] been locked up or not? These are all things that you can absolutely simulate and and create by condition. But what we've also seen is that these models behaviors. And so, you'll have partners and you know, uh companies like Skilled [30:54] AI. They have this wonderful video. They they effectively had a robot dog and then they locked up one of the knees. And by locking up the knee it actually behavior actually learned to adapt on how to walk uh on three working uh legs [31:08] hobble on the other? And we're starting to see this with more complicated systems as well with these humanoids where an ankle um I was visiting one and we they couldn't quite figure out what was wrong with one of their units. [31:21] funny." And it turns out that the ankle joint was slightly broken. And the policy itself had already adapted to this and so, it was it wasn't walking >> Yeah, it took it into Yeah, and so, it it learns because [31:36] you know, uh you're you're trying to um train these things to to walk. You're uh regardless of what's happening with the rest so it's going to, you know, >> Um ideally you don't have to because that [31:50] itself. >> Now, let's dream a bit. And like due to software and hardware optimization, we had a 1 million x ASB that in 10 years, [32:03] which is just just insane. If we had another 1 million x in the next 10 years, what do you think could we do that we cannot do now? one of the things I like to think about is [32:19] learning, each of the when you you see the the scene, you have um an environment and you got a bunch of clones of this robot right? They're all trying to do the same task, but they're doing it slightly [32:33] bit differently. You can kind of you can just think of that as experience. It's a little bit like life experience. So, for a human, we have we have to go at at 1 x, right? Every moment is a is an experience for us, but we can only step [32:48] through a moment a second at a time. For these robots, if you're if you're able to train these policies faster than real time, which is what we're, you we could do in the future with, you know, a 100 million x speed up, is you [33:03] can train these policies to do extremely complicated behavior in a matter of years. All right, so this experience that we're giving it is really amplified with the more the [33:15] efficiency that you have in the ability to take take advantage of these systems. The other thing that enables it will likely enable is when you have a million times speed up over a large system, it's another way of [33:29] over a small system. Well, these small systems are going to allow you to adapt things that humans are are, you know, crazy good at is if I teach you a into a novel environment, like a you know, you've never seen this environment [33:44] not something you're familiar with, you know, I send you to Alaska and say, fishing boat and you got to go learn how to how to carve up the fish, you know, like the fisherman." And you've only done it at home and you you know, you're [33:58] here and and maybe I just have no idea how you're going to do at edge learning, There's no there's no going to school, no internet. Um these are things that could actually do training on the robot itself instead of having to send it to [34:13] want to do large-scale batch training, you're sending it up to the cloud. Um as we get further along, not only are robots going to have the ability to do going to start working on super human complicated tasks. And these tasks they [34:28] would actually be able to have it running these simulations, you know, on >> Continuous learning. You know, we we have to reason in the situation, right? >> When you look at at um I'm not good at Rubik's Cubes, but I've [34:43] at and they >> Proud Hungarian product. Just saying. the world and and finally have you seen some of the models are actually able to quite quickly in seconds and and it's a even with agents just streaming it over [34:58] that's actually been really fascinating, but the and then to imagine the problem is actually us putting simulation in our constantly simulating. If you're an athlete, you see them focusing and [35:12] they're they're trying to imagine and project in their mind, okay, what am I body? How am I going to do this? You know, you're in the Olympics and and thinking about what they're about to do. That's just simulation in your head, [35:25] robot? It an ability to simulate, you know, on edge in a sense. So I I think >> X on you could do that. >> Yeah. Yeah. >> Yeah, love that. Love that. When I was at the Nvidia headquarters, I [35:38] heard about a robot that was going around handing out popcorn to people. I haven't seen it, but is that true? >> Yeah. Yeah, we have Well, we have them in in the gear lab and uh [35:52] popcorn. This thing is going all the time. Um what we're seeing is uh robots are now finally at the stage where the policies and the hardware are able to be deployed into [36:05] useful things and and I think that handing popcorn is actually quite useful >> They love it. >> I know and I was I was always promised deliver me popcorn to my cubicle and so I'm I'm [36:20] that can go up the stairs because when we had an AMR it was yeah. been hoping for for a long time, but I'm not on the first floor when I first joined and so a wheeled robot was going to have trouble getting up the stairs. [36:33] popcorn, you know. Now we now we actually could have this going around and and I we don't have it yet. Um but we're we're only a click the >> the the the amazing part of the age that [36:46] we live in is there was this there was a time a few it's going to take us 10 years to solve some of these problems. And with the they've advanced and you know, over the last [37:00] we didn't have agents in January of this year. And now it's it's everywhere. Um and we're not even totally at the end of the year. Um we've now taken a 10-year horizon and just crunched it down into three to five, right? And really the [37:15] going to have the hardware systems that are capable of enacting these policies? Do you have full body dexterity that can do all the types of things that that we have enough compute to actually train up the policies to be able to do the [37:28] complicated task that we want. So, it's close. >> What kind of problems are still there that are easy for humans but difficult for machines? And how do you address [37:42] >> So, this is you know, we're we're so close but somewhat far. It's Um I forget there's a paradox where things that are easy for humans are hard for machines. And things like um in-hand manipulation [37:59] can be quite difficult. Things like um assembling LEGOs, you're going to start can start, you know, really accomplishing. But imagine some of the learned to use our body to do micro-assembly. These are such hard [38:15] describe. I need you to take the screwdriver and you need to put this here and you got to certain locations but the dexterity required is so complicated for a robot. And that's that's something that we we need to [38:27] start solving for but it requires a few things. First, you have to have um either real data uh or simulated data or a simulator that's able to actually um that is simulate this this complicated behavior. Second, you have to be able to [38:40] train up a policy that's able to do do all of the priors, you know, can it use sub-millimeter precision? Can it do all of this? And so you're building your way up a little bit like you're watching a kid grow and you have to do this on, you [38:53] time. And then the last is do you have the hardware that has that precision? I think that that's somewhat overlooked. We've had automation, high-level precision automation for for decades. And if you look at the robots today, [39:07] they're able to do gross manipulation and they're starting to get really good when we think of extremely high precision, you know, 0.003 mm. demonstration data is not going to be great for it. Um but we'll train a [39:22] that's actually able to do this as well. And so the hardware is starting to come see is there's now options and there is plenty hardware builders and OEMs that are actually building systems that we can [39:36] now start really, you know, taking this this next step. So I think those are the hard things are easy for us. Um but are harder for robots. Um on the other hand, um you know, if you ask it to do math or, you know, try and generate millions [39:49] of of uh sample data, uh it's going to be almost impossible for us, but you it in a minute. >> I wanted to ask your opinion about this. Um do you feel that like first let's think strictly LLMs. So [40:03] not even physically I like would AI at some point become good enough? at some point become good enough? Or do we always want the best? And the question is deceptively difficult because my [40:18] impression is that yes, at some point it's going to be good enough. Like it's not a nice way to measure things, but let's say that each LLM has an IQ. intelligence is a vector quantity and maybe even more complex, but just just [40:33] for simplicity. And at some point you're going to have instant, that has an IQ of, I don't know, 170. And there's a new one that's 220. And at some point you say like who cares? Like any of these will do. [40:49] So for me, it's easy to think that yes, at some point it's going to be good enough. But at the same time, I I try to think whenever I ask something, like which AI do I use? Do I use the the thinking AI or the instant [41:02] AI? And I I feel that I I reach out to the thinking AI a lot more than I think. Like if I have a contract to be reviewed, like like I understand that the instant is good, but I I kind of [41:15] questions, questions about, I don't know, medication or or something. Like LLMs should be maybe just one data point, like not a doctor, all that. But still, if I'm asking something, I know the [41:28] want the best. So what what do you think? How do you find yourself using >> I'll answer it and I can in in in this answer I'll apply it to both LLMs and and physical AI models. When when you have a sufficiently Okay, [41:43] I'll separate into two two buckets. The first is generalized models and the second are specialized models. If you're in a medical field or or in Um having a specialized model that gets you [41:56] to, you know, 170 or 200 IQ, whatever we whatever we think is an amazing baseline that gets you across the line for, you know, 80 90% of the problems. I think that kind of good enough is actually what people are going [42:10] to be using day-to-day. But we are always going to be pushing the frontier to solve for. One of the things that humans are good at is we're we're great at inventing work. You know, we we've invented many means of automation and at [42:22] actually just found more ways to make ourselves, you know, push the the frontiers of of automation, we push the frontiers of the things that we can make, the things that we can create. The types of challenges that we want to [42:35] know, we're we're here on Earth, but at some point you want to, you know, go further. So the baseline intelligence will be great for us to be able to raise everybody up to be able to accomplish, you know, a [42:49] a certain let's say it's the predominant common or the the general amount of which I think is is perfect. It's like going to get a master's degree is really getting a PhD is not necessarily for everybody but it is for a very specific [43:05] for our audience. >> Especially for our audience. >> The fellow scholars. Always wanted to say that. >> [laughter] >> And the general model is once you hit [43:17] that baseline I think good is actually going to be what's in intelligence is what's always going to be having us push forward. So we're right now we're still just trying to get to the baseline of physical intelligence [43:30] point where these these frontier models are just becoming so excellent in a certain distribution of tasks that most people will be okay with you know what was frontier today might [43:43] be just common tomorrow. But there's also going to be an immense amount of frontier for for pushing everything else. >> All right, two of the hardest questions. [43:55] One, favorite Two Minute Papers episode. You know, I would have come prepared I would have remembered the paper names. would have remembered the paper names. The you know, I I think that in general [44:08] I love your uh when you go through fluid fluid and of parts. One they're they're just visually appealing but two it's not my understand it is actually when I'm watching it at lunch. [44:22] It's always wonderful to see my colleagues papers. And there you know, motion generation and you've done some really wonderful coverage of of our motion generators and and what we can actually do with taking human [44:37] human motions and applying it to to these avatars and and simulation and in the future you're going to see how that immediately translates to the physical Man, you got me there. >> No, that was that was really good. I'm [44:50] very pleased with that. Um last question. This is the hardest. How do I get you to answer my emails? >> [laughter] [45:02] that at symbol works really well. The my I'm learning right now to have my agent actually help me undo Let's just say I'm on a lot of automated >> Tell your agent that Two Minute Papers is priority one. [45:16] >> priority one. Honestly, I think you should The easiest is you call me. >> All right. All right, Spencer. I've learned a lot. And and when I go home and I cut this video together, I'm going to watch it, you know, five, six times [45:28] more and I'm going to learn even more. So, this was amazing. Thank you so much. So, this was amazing. Thank you so much. All right. We're done. Thank you. >> [laughter] >> That was That went by in a flash. [45:42] >> Ah, tricky tricky last question there. >> I use Lambda to reproduce AI research papers often in minutes. It's also great to train your own models or fine-tune an existing one. Run inference or text to image or video easy peasy. Running a [45:58] deep seek chatbot or agent super fast, super reliable. Lambda gives you powerful Nvidia GPUs to run your own experiments. I test ideas from the papers I cover and moments later, results. Love it. Seriously, try it out [46:15] results. Love it. Seriously, try it out now at lambda.ai/papers.