NVIDIA's New AI Shouldn't Work... But It Does

[00:00] Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. We’ve been trying out doing the videos with a  camera. I really enjoyed it, and your feedback   was also absolutely incredible. I’ve never  seen anything like this. So many comments,  

[00:17] thank you so much for the kind words everyone.  So, we will try to do more of this. But note   that this one is a classic voice paper that  we’ve always done here. It was done before   we did the camera thing, I thought I would  record this little intro now so you don’t  

[00:33] get surprised. And then next video I’ll be back.  And for now, please enjoy this super fun paper. How do robots learn how to be a good robot?  Well, surely not like this. Haha. Not by just  

[00:47] running around in the real world. Of course!  I mean, imagine a real robot doing this for   years and years. It would be dangerous to others  and itself. So here is a better question: how  

[01:00] do we teach a robot to be a helpful, good robot  safely? Well, we put it inside of a video game. we simulate physics, and let it fail.  A lot. Then, over time, get better.

[01:15] Now, I’ve been to a bunch of AI  and robotics labs around the world,   things work fantastically well in a simulation,  and then, when you put them into the real world,  

[01:29] huge disappointment. Something that worked really  well suddenly does not work well or at all. Why? Well, the main reason is that  simulations are often just not good  

[01:41] enough. They often mimic reality, but  they are not a substitute for reality. So what do we do? Well, of course, try  to use reality. In this work, DreamDojo,  

[01:53] scientists said okay, let’s feed the AI 44  thousand hours of videos of humans doing stuff. That sounds great, except the fact  that it is completely useless. 

[02:06] Why? Well, humans and robots have  entirely different physical bodies,   hands, and joints. Also, the video does not  contain action information. It’s just a soup   of data that doesn’t say what joints  are exerting forces and how. Nothing.

[02:23] Well, they propose 4 genius ideas,  and I hope that will make this work,   One, if the video does not have labels  on what actions are taking place, well,  

[02:38] then let the AI try to understand it  and make up its own story of what is   happening. If you see someone waving  at a bus that is pulling away. You   don’t need a text label to know that  someone has just missed their ride.

[02:53] Two, this dataset is stupendously large. It  has more than 4 billion frames, and probably   more than 1 quadrillion pixels. Okay, that is  too much information. It is almost impossible  

[03:07] to handle. So the AI has to learn what is  important and what isn’t. How? Well, it is   forced to compress information. A musician does  not need to know every song in the universe. They  

[03:21] have to know that there are 12 notes in a scale,  and every song is just built as a combination   of these fundamental notes. This forces the AI  to look at only the most critical information.

[03:34] But guess what, it is still not enough to just  dump videos into the robot and make it work. Why? Well, three, if you train a robot to pick  up a cup at a global position, it learns to  

[03:47] reach for that exact spot in the world. That’s  no good. Why? Well, if you move the cup a few   inches to the left, the global coordinates change  entirely, and the robot has no idea what to do.

[04:01] So, what scientists said, instead  of using absolute robot joint poses,   let’s transform the inputs into relative actions.  If you are cooking, sometimes you don’t need  

[04:13] absolute coordinates. Here, the knife only needs  to know where it is relative to the carrot’s spot. And believe it or not, this is still not  working. We need something more. What do we need?

[04:27] Well, four, the goal is that the AI learns  cause and effect. Jelly bunny hits the wall,   and something happens. Try to learn  that by predicting the next frame.  

[04:39] The problem is that the AI is cheating. Like  a student, it just looks at the solution at   the end, and says, oh yeah, I was gonna say  exactly that. So how did they prevent that?

[04:52] Well, they fed it actions in  small blocks of 4 at a time,   so it cannot cheat by peeking at the  future to guess what happens right now. Okay, this was a lot of genius stuff,  so it better give us something amazing.

[05:09] Let’s see what we got. Previous method.  Can’t predict the future…oh my, look,   that hand clips through the piece of paper.  Now hold on to your papers Fellow Scholars for  

[05:22] the new method and….oh my! Look at that!  The paper finally crumples beautifully! And with previous methods, the clipping gets  even worse. Look. That’s not predicting reality,  

[05:37] that’s just guessing. New technique  - now we’re talking! Looking good! and the lid refuses to move. No good. New  technique, the lid moves! Woo-hoo! Yes,  

[05:57] this is the corner of the internet where we  get unreasonably happy about a moving lid. the new technique is so much better than  previous methods. This is a huge leap forward!

[06:12] Now, this gets even better. So it finally  understands the world better than previous   techniques. So what do we pay for this? How  much slower is this than previous methods? Well, it is pretty slow because it requires 35  

[06:27] heavy denoising steps just to generate  one prediction. But wait, don’t despair! We can use distillation here. Distillation is  a training phase where a fast student model  

[06:39] is used to learn the predictions of the  slower, high-quality teacher model. The   goal is that the student would be nearly as  good as the teacher model, but much faster.

[06:51] Well, let’s test that! Oh my, now the  student is a heck of a lot faster.   It seems that it is 4 times faster than the  teacher that was used to train it. It runs at  

[07:05] about 10 frames per second. Understanding the  world and predicting how it will change at a   speed that is interactive. That is absolutely  insane. Well done! And the kicker is that  

[07:19] they also predict very similar outcomes.  This is an absolute slam dunk paper. Wow. Now for you wise Fellow Scholars out there, I’ll  note that we talked about a technique called NeRD,  

[07:32] Neural Robot Dynamics. That was a robot  AI that trained in its own imagination.   So how does this relate to that? Now NeRD  was building a perfect 3D environment. This  

[07:45] one thinks in 2D. It just sees the world as a  bunch of 2D video pixels on a flat TV screen.   Thus, this one is able to learn about  thousands of everyday objects. So cool!

[07:59] and robots that we can all own ourselves.  In a world full of subscriptions,   it is so refreshing that we get all of this  for free. A ton of code and pre-trained  

[08:13] models are available for free for all of us.  No silly subscriptions and proprietary code. A free brain that you can upload to your own  devices and use it however you want. Love it.

[08:25] So this finally puts us one step closer  to having a robot fold our laundry,   or cook a healthy meal. Or help a  specialist doctor perform surgery   from the other side of the planet via  teleoperation. What a time to be alive!