It is an educational repository with a purpose to understand how Stable Diffusion and Diffusion models work.
Why did I decide to create this repo?
I started this project with the purpose to understand how Stable Diffusion works and what math it uses. I've read a lot of articles with math explanation and code implementation, but I didn't find something which bridges them in the one place.
Do you need to have a degree to understand math?
It depends on level you have, but I tried to write formulas with all important details and links to resources, which you can read if you get stuck.
Latent Diffusion models based on Diffusion models(or Simple Diffusion). It's the heart of Stable Diffusion and it's really important to understand what diffusion is, how it works and how it's possible to make any picture in our imagination from just a noise. These are my suggestions about steps to understand the information.
-
Start with Math Explanation.ipynb . This is my explanation of math behind diffusion models, based on great articles of other authors and of course my own research. It took two weeks to understand all details of formulas, but I tried to give the information easily.
-
In this repo you can look at implementation of two different diffusion models: Conditional Diffusion and Simple Diffusion. The Conditional is similar to Stable Diffusion, but it works only number(number means class or digit) input, not text prompts. So it's better to start from Simple Diffusion models.
-
For experiments I have created two jupyter notebooks:
- For training and sampling Simple Diffusion
- For training and sampling Conditional Diffusion.
To start working with code, please download all required dependencies:
pip install -r requirements.txtI put all models with git lfs, so to get all models, just run:
git lfs fetch --allIf you want to get random samples, open Simple diffusion. Or if you want to control a generation with input as class, open Conditional diffusion.
All jupyter notebooks have PATH_TO_READY_MODEL, which you should fill to skip the training phase.
If you want to train a simple diffusion, open Simple diffusion, choose dataset (cifar10 or mnist) and run all cells.
If you want to train a conditional diffusion, open Conditional diffusion, choose dataset (cifar10 or mnist) and run all cells.
| Model | FID (CIFAR 10) |
|---|---|
| Original Diffusion | 3.17 |
| My Diffusion | 30 |
- https://lilianweng.github.io/posts/2021-07-11-diffusion-models/
- https://calvinyluo.com/2022/08/26/diffusion-tutorial.html#mjx-eqn%3Aeq%3A79
- https://arxiv.org/abs/2006.11239
- https://arxiv.org/abs/2207.12598
- https://github.com/TeaPearce/Conditional_Diffusion_MNIST
- https://github.com/cloneofsimo/minDiffusion
- https://theaisummer.com/diffusion-models/
- https://yang-song.net/blog/2021/score/