DeepSeek's AI Efficiency Fix — Full Breakdown & Transcript

[00:00] Scientists at DeepSeek have invented something amazing and exactly at the right time when we need it most. You see, we are entering the age of AI, but I am really surprised.

[00:12] I just found out that the way these AI systems run on our computers is incredibly inefficient. So if you want your AI assistant to answer quicker, you need more compute power, clear as day,

[00:26] but you may find that as you add more compute, it does not get faster. But how can that be? You know, it's kind of shocking, given that companies are paying billions and billions of dollars

[00:38] for more compute to run these AI systems. How is this possible? Imagine reading a book, and now imagine that every time you turn the page, you forget about the characters. That's not a great way to read books, right?

[00:52] Here is what happens in practice. As soon we have a huge brain, the size of a mountain, and we want to talk about a book. If the book is one page, we just memorize that one page and just talk about it quick and easy.

[01:07] Now imagine that the book grows. It is now huge! And since we forget about everything, the moment we turn the page out. If we want to talk about it, we have to read it all the time.

[01:20] So, our brain is huge and hungry, but there is a problem. Information is coming in through a straw. So then, we spend most of our time not thinking, but reading slowly.

[01:33] And that is exactly what the graphics cards of today are doing when you run an agentic AI system on hard problems. All those billions of dollars sitting at 40% utilization.

[01:48] This is a horror story. That's a tough problem. So, what is the solution? Well, of course, you don't need all those GPUs. So send them to me. Problem solved.

[02:02] Okay, so, how did scientists at Deepseek solve it? Dear fellow scanners, this is two-minute papers with Dr. Kano-Jone Fahir. Now, of course, they say you don't need a bigger brain.

[02:14] You need a bigger straw. So, in today's systems, there are AI chips that do the reading. We call them pre-filled machines. They are the straws and they are completely jammed,

[02:26] but there are also different kinds of machines in the network, the decoding machines. And their straws are nearly completely empty. They just sit there often unused.

[02:38] So they say, use those to do the reading and have it take a second path to the pre-filled machines. Finally, it's a clever detour that less the brain do its job,

[02:50] but there is a problem. This shortcut takes the same high-speed roads that the AI needs for thinking. If we don't do this well, hooray! We solved the traffic jam. And when they ask us, how?

[03:03] Well, by introducing another traffic jam. Okay, so, what is the solution for that? Well, traffic control. On these roads, thinking traffic gets priority.

[03:16] Memory traffic, however, gets left over space. This is absolute genius because it does not give you more compute. No, it gives you access to the compute that you already have.

[03:29] Okay, so, what is the key result? Well, hold on to your paper's fellow scholars because it speeds up this whole network from 40% utilization to about 80% utilization.

[03:41] In practice, almost twice as much work from the machine you already bought. That is an insane jump in just one paper. I am completely stunned. And the main use case for this is when you have long, multi-turn,

[03:56] agentic workloads. And they give this technique away for all of us for free forever. Whew, now it is not a magic bullet for all AI agents to run twice as fast.

[04:08] No, no, it is situational, but it helps exactly in the hardest situations where we need the most. Long conversations, lots of data. That's when things really slow down.

[04:20] Also, note that this is not a shiny new AI system that you can easily write headlines about. It's not the brain. It's a better road system to the brain. It's something that you implement in a data center when you serve these AI systems.

[04:36] So, you don't see a lot of headlines on this because it's not the shiny thing that is easy to sell, but it is absolutely brilliant. And I really wanted to show it to you. And all of us get value out of this kind of open science.

[04:50] If this idea makes it to real serving systems, it might lead to cheaper AI inference for all of us in the future. And they don't close it down and keep this knowledge to themselves.

[05:02] They give it all to us as a gift. How cool is that? That is the power of the papers. What a time to be alive. A word of optimism. Enjoy in a world where you hear about doom coming from every direction.

[05:16] Subscribe and hit the bell if you enjoyed this. Here you see me running the full deep seek AI model through Lambda GPU Cloud, 671 billion parameters running super fast and super reliably.

[05:31] This is insane. I love it. And I use it on a regular basis. Lambda provides you with powerful Nvidia GPUs to run your own chatbots and experiments.

[05:43] Seriously, try it out now at lambda.ai slash papers or click the link in the description.