Diffusion Models, Simply Explained

Diffusion models are generative deep learning models that have skyrocketed in popularity in recent years due to their exceptional performance, beating out other established generative models like the Generative Adversarial Network (GAN) and Variational Autoencoder (VAE).

Diffusion models are most known for their image generation abilities—they are used in popular generative models such as DALL-E or Stable Diffusion.

Even though diffusion models were introduced in 2015, they didn’t start taking off until early 2020, when Ho et al. released a paper proving how probabilistic diffusion models could be effectively applied to image synthesis.

That’s great, but how do they work, and what even is diffusion? Diffusion refers to the phenomenon of a “scattering” of particles, and it is widely studied in the field of thermodynamics.

An example of this is sunshine that passes through clouds, creating rays that scatter and spread out in different directions. Diffusion is occurring from the random motion of the light particles.

This natural process is similar to what happens in diffusion models used for image generation. Random noise is added to an input image, progressively showing scattered pixels of colors that differ from the original image.

Specifically, diffusion models add Gaussian noise to training data successively, which we refer to as the forward diffusion process, then learn to reconstruct the data by reversing this noising process—which we refer to as the reverse diffusion process.

• Essentially, diffusion models add noise to images, then learn to remove the noise they added.

After training, diffusion models will be able to generate high-quality images starting from just random noise inputs, iteratively denoising them to create full images of anything the model was trained on.