I read some papers and blogs, but still feel confused about the mathematics of diffusion models. So I decided to write down some formulas to clarify my understanding. In particular, I want to clarify the connection between discrete and continuous diffusion models, and how to derive the loss functions, with consistent and clear notations.
The material in this note may not be the state-of-the-art, but it’s enough for me to understand the basic ideas. Hopefully, it can help you as well.
Check out DDPM, SODE perspective, and their connections.