diffusion for beginners

• implementation of diffusion schedulers with minimal code & as faithful to the original work as i could. most recent work reuse or adopt code from previous work and build on it, or transcribe code from another framework - which is great! but i found it hard to follow at times. this is an attempt at simplifying below great papers. the trade-off is made between stability and correctness vs. brevity and simplicity.

$$\large{\mathbf{{\color{green}feel\ free\ to\ contribute\ to\ the\ list\ below!}}}$$

• dpm-solver++(2m) (lu et al. 2022), dpm-solver++: fast solver for guided sampling of diffusion probabilistic models, https://arxiv.org/abs/2211.01095
• exponential integrator (zhang et al. 2022), fast sampling of diffusion models with exponential integrator, https://arxiv.org/abs/2204.13902
• dpm-solver (lu et al. 2022), dpm-solver: a fast ode solver for diffusion probabilistic model sampling in around 10 steps, https://arxiv.org/abs/2206.00927
• heun (karras et al. 2020), elucidating the design space of diffusion-based generative models, https://arxiv.org/abs/2206.00364
• pndm (ho et al. 2020), pseudo numerical methods for diffusion models, https://arxiv.org/abs/2202.09778
• ddim (song et al. 2020), denoising diffusion implicit models, https://arxiv.org/abs/2010.02502
• improved ddpm (nichol and dhariwal 2021), improved denoising diffusion probabilistic models,https://arxiv.org/abs/2102.09672
• ddpm (ho et al. 2020), denoising diffusion probabilistic models, https://arxiv.org/abs/2006.11239

prompt: "a man eating an apple sitting on a bench"

dpm-solver++	exponential integrator

heun	dpm-solver

ddim	pndm

ddpm	improved ddpm

* requirements *

while this repository is intended to be educational, if you wish to run and experiment, you'll need to obtain a token from huggingface (and paste it to generate_sample.py), and install their excellent diffusers library

* modification for heun sampler *

heun sampler uses two neural function evaluations per step, and modifies the input as well as the sigma. i wanted to be as faithful to the paper as much as possible, which necessitated changing the sampling code a little. initiate the sampler as:

sampler = HeunSampler(num_sample_steps=25, denoiser=pipe.unet, alpha_bar=pipe.scheduler.alphas_cumprod)
init_latents = torch.randn(batch_size, 4, 64, 64).to(device) * sampler.t0

and replace the inner loop for generate_sample.py as:

for t in tqdm(sampler.timesteps):
    latents = sampler(latents, t, text_embeddings, guidance_scale)

similarly, for dpm-solver-2,

    sampler = DPMSampler(num_sample_steps=20, denoiser=pipe.unet)
    init_latents = torch.randn(batch_size, 4, 64, 64).to(device) * sampler.lmbd(1)[1]

and, for fast exponential integrator,

    sampler = ExponentialSampler(num_sample_steps=50, denoiser=pipe.unet)
    init_latents = torch.randn(batch_size, 4, 64, 64).to(device)

and, for dpm-solver++ (2m),

    sampler = DPMPlusPlusSampler(denoiser=pipe.unet, num_sample_steps=20)
    init_latents = torch.randn(batch_size, 4, 64, 64).to(device) * sampler.get_coeffs(sampler.t[0])[1]

soft-diffusion

a sketch/draft of google's new paper, soft diffusion: score matching for general corruptions, which achieves state-of-the-art results on celeba-64 dataset.

details can be found here