---
title: 'Solved: The Bug That Haunted AI Video For Years'
source: 'https://youtube.com/watch?v=yzajLZXh9JU'
video_id: 'yzajLZXh9JU'
date: 2026-07-24
duration_sec: 575
channel: 'Two Minute Papers'
---

# Solved: The Bug That Haunted AI Video For Years

> Source: [Solved: The Bug That Haunted AI Video For Years](https://youtube.com/watch?v=yzajLZXh9JU)

## Summary

This video explores a breakthrough in AI video generation that addresses the persistent issue of unnatural motion. The presenter, Dr. Károly Zsolnai-Fehér, explains how a new technique identifies and removes 'bad influences' from training data, dramatically improving motion realism without requiring more compute or data.

### Key Points

- **AI Video Generation Capabilities** [00:00] — AI can now generate high-quality videos from text prompts with exceptional controllability, producing different movies that end the same way.
- **Photorealism vs Motion Problem** [00:41] — AI excels at photorealism but struggles with motion; frames look right but movement feels wrong.
- **Compute Scaling Test** [01:10] — Increasing compute by 4x and 32x improves motion quality, but the problem persists.
- **Identifying Bad Training Data** [02:26] — A new technique asks the AI where it learned motion, revealing that cartoons teach conflicting physics (e.g., characters pause mid-air).
- **Cutting Bad Influences** [03:43] — Removing bad training data and fine-tuning with good data dramatically improves motion, e.g., a coin spinning correctly.
- **User Study Results** [04:27] — The new method achieved a 74.1% win rate over the original in a user study with 50 videos and 17 participants.
- **Motion Masking via Optical Flow** [05:14] — They separate motion from appearance using optical flow, applying the mask to internal learning signals, not the video itself.
- **Compression with Johnson-Lindenstrauss** [06:14] — To handle billions of parameters, they compress learning signals from over 1 billion to 512 numbers using Johnson-Lindenstrauss projection, preserving relative distances.
- **Broader Lesson: Quality Over Quantity** [07:52] — The main message: a tiny clean signal beats a mountain of junk. Verify information and take in less but better.

### Conclusion

The paper demonstrates that improving AI video motion doesn't require more data or compute, but rather curating training data to remove misleading sources. The technique of compressing learning signals enables efficient identification of bad influences, offering a powerful lesson for AI and human learning alike.

## Transcript

Today, generating eye-poppingly high-quality  videos just by writing a text prompt is possible.   You can also get exceptional controllability  as well. You can generate three movies that  
look completely different,  but land on the same ending.   Almost anything you can think becomes  achievable, effortless and inexpensive. Now, how they are kinda taking over the internet  
is another story. But pretty much all of these  systems have a huge problem. What is the problem? Is it issues with photorealism? No. In  photorealism, these AIs are second to none. I am  
a light transport researcher by trade, I like to  write programs that create photorealistic images,   and I feel that many of their results are nearly  impeccable. I spent more than a decade to learn  
this craft, and these AI systems are picking it up  at an incredible speed. That is absolutely crazy. But, not so fast. What about motion? Well, now  we got a problem! Yup, motion breaks the spell.  
The frame looks right, but the movement feels  wrong. And at this point, most AI researchers   at this point say, no problem. Just give it more  training data, and more compute, and we are done.
Let’s actually test that. This is the base amount  of compute for OpenAI’s Sora from two years ago.   Base amount of compute. Yuck. This is  not great, and if you look closer…I  
think you shouldn’t, you notice that  this is what nightmares are made of. Now, if we add 4 times more compute, we get this.   Perfect? Not even close. But  the trend is shouting at us. 
Now, with 32 times more compute, we get this.  Now we’re talking. The result starts to sing. So, case closed, right? If the motion is not  good, and if you don’t have more compute, because  
who does these days, well then, let’s add more  training data. Let it look and learn some more. Except that this is completely wrong. That is  what this paper is about. When we see an AI  
generate motion, they developed  a technique that is able to ask,   okay little AI, where did  you learn that? I love that! Let me give you an example. A foam cube floating  on water. And it gives us waves crashing over a  
pier, surfing, splashing ocean waves. This is so  cool! So this is where the knowledge came from.   But wait, they say that if these are  positive examples for your learning,  
Oh! This makes sense - these really are the worst  for learning. Why? Because cartoons, for instance,   teach completely conflicting information about  physics. In cartoons, characters pause mid-air  
before falling, maybe even holding a tiny  little umbrella. Bodies bounce like rubber,   and snap back into their original  shape a moment later. Fun for us.  
Not so fun for an AI model  trying to learn real physics. Wait a second…I have an idea. What if we don’t just put in there more   training data. What if we give it less? Cut  out those bad influences! Can it do better?
Let’s try it out together. Yes! With the base  model, we get a coin which is spinning around   the wrong axis. And now, hold on to your papers  Fellow Scholars, because here comes the magic.  
After cutting out these bad influences  and fine-tuning the AI with the good ones,   look at that! That is a beautiful spinning coin. I got to say I was a bit less impressed by  the ball example, yes the new one is better.  
We have seen plenty of systems pull off this  kind of movement. In any case, we are Fellow   Scholars here, we don’t hand out medals for a  couple cherry-picked examples. No. We are more  
rigorous than that. We look at the research paper.  Does the paper deliver? Oh yes, yes it does! I look at the user study, and see that it lands  the punch. They asked people to judge whether  
the new or previous method was better. They did  it across 50 videos and 17 participants. That   is 850 little tests. And…drumroll, it has a 74.1%  win rate over the original. That is stunning.
Okay, so how on earth did they do that? Can  we catch and AI in the act of remembering?   Is that even possible? And what does  that mean for us? Dear Fellow Scholars,  
this is Two Minute Papers with Dr. Károly  Zsolnai-Fehér. Now that’s a late cold open. Alright, they did two things to ensure  that this concept works properly. One,  
you need to be able to separate how things  move from how they look. To do that,   they introduce a motion masking step  through a technique we call optical flow.   An old idea. Works great for tracking the  path of points over a video. Good call.
But here is the genius part. They don’t  apply this mask to the video itself. Nope!   Instead, they apply that mask to the  internal learning signals of the AI.  
Genius idea, yes, but unfortunately,  two, there is a huge problem with this.   What is the problem? Modern AI models have over 1  billion parameters. Storing and comparing the full  
learning signals for thousands of videos  takes too much computer memory and time.   That’s crazy town. Not feasible.  Instead, they found a way to   get this, compress down these more than a billion  numbers into, excuse me? Am I seeing correctly? 
That’s right, 512. Down from more than a  billion. And the results are almost the same. Wow! That is insane. The technique they use  is called the Johnson–Lindenstrauss projection  
and it was used in Google’s TurboQuant  compression algorithm as well. That is   one to ease the memory constraints of large  language models on your GPU. What does it do? What it does is it shrinks  high-dimensional data into a tiny space,  
but in a way that it preserves the  relative distance between these numbers.   Picture a wooden chair. Now picture its  shadow on the floor. The chair lives in 3D.   The shadow lives in 2D. The shadow needs much  less data. And if the scene is set up right,  
the distance between the four chair legs remains  the same. And that means that this projection   allows us to retain important properties  of the data, but cut away a lot of fat.
And all this is put together to achieve  one thing: to be able to find what videos   influenced the AIs decisions. And then,  to cut away all the junk knowledge. And that is also super important  for our thinking. You see, there  
are topics where I hoped that the more I read,  the smarter I would get. Read more, grow wiser. Not true. There are many areas where the more I  read, the more I found that I just got stupider.  
It took me years and years to find out that there  are topics you can read and learn all you want,   if the quality of information is low. It  does not educate. It deforms your thinking.
So what is the solution? You need to be able to  separate the real from the fantasy. You don’t need   more. You need less, and you need better. Like  you saw in the paper, truth is the best teacher.  
This technique just showed a tiny clean  signal beats a mountain of junk. Slow down,   don’t take everything in. Try to verify what you  actually hear, and try to take in less. To me,  
that is the main message of this paper.  Brilliant work. Brilliant lesson. Love it.   And they promise that we’ll get the  code for free. What a time to be alive!