---
title: 'How Diffusion Models Work'
source: 'https://youtube.com/watch?v=tp69sZPXexQ'
video_id: 'tp69sZPXexQ'
date: 2026-09-02
duration_sec: 176
channel: 'Steve Mould'
---

# How Diffusion Models Work

> Source: [How Diffusion Models Work](https://youtube.com/watch?v=tp69sZPXexQ)

## Summary

This video explains how diffusion models, like those used in AI image generators, are trained and how they generate images. It breaks down the process of adding noise, training the model to predict it, and using CLIP to guide the output.

### Key Points

- **Training with Noise** [00:00] — Training involves adding random noise to images and having the model predict the noise, then adjusting parameters based on errors.
- **Generating Images** [01:09] — After training, the model can remove noise from pure random pixels to generate recognizable images.
- **Learning Features** [01:50] — The model learns features like edges, furriness, and styles through multiple iterations.
- **CLIP for Guidance** [02:16] — CLIP aligns text and image features in a shared vector space, allowing the model to generate images based on text prompts.

## Transcript

The way you train a diffusion model neural network is you take some images, that's the training data, and you tweak the RGB values of all the pixels randomly, that's called adding noise.
You then ask the model to tell you how it thinks you tweaked all of those pixels. Like, I think this pixel is too green by this much, this pixel is not red enough by a small amount, and so on.
And it gives you that as an array of values. In other words, you're asking it what the noise was that you added to the image. And remember, this is still the training stage. So at this point, you say, no, that's terrible what you came up with.
This was the actual noise that I added. Now go away and fix all of the parameters in your model so it does a better job next time. But actually, you're doing it with millions of images at the same time. So you're saying, look, you got all these pixels wrong by this much in this image.
And you got all these pixels wrong by this much in millions of images. tweak all of the parameters inside your neural network so that it's better for all of these images at the same time. I told you it was wild. So anyway, you train it on noisier and noisier images.
In other words, images where you've messed about with the values of all the pixels more and more and more until eventually you can give it an image that's just pure noise, completely random pixels. And so then when it gives you a prediction of the noise that was added to an image,
when you remove that prediction it will give you an image that looks like something recognizable from the training images Now at this point you have no control over what comes out at all It just gonna be random what you get
Well, the model is able to figure out what noise needs to be removed from an image because it has learned to understand features of images. And in some ways, it's very similar to the way a language model understands the meaning of text,
which is to say it goes through several iterations where its understanding of image features become more and more nuanced. So maybe in the first iteration, it's just picking up on where the edges are in the image.
And then in later iterations, maybe it's picking up on concepts like furriness or maybe light glinting off a shiny surface. And then later on, maybe that it's in the style of a Monet. It would just be a long list of values that represent these things.
And again, if you actually interrogate the values, it might be very difficult to actually pick out specific things that we would understand. And that's where a model called Clip comes in. Clip was trained on millions of images from the internet and their captions.
So during training, it was learning the meaning of the text and the features of the image and the vectors it was producing for both of those things. It was putting in the same place in a shared vector space of semantic meaning and image features.
And so the model will now guess what noise needs to be removed in a way that pushes the image towards having features that are encoded in this vector.