Diffusion Scores
Chapter 17: Score Function and Energy-Based Models — Diffusion Preview
From the book
Chapter 17: Score Function and Energy-Based Models. In the chapter mind map this icon labels Diffusion Preview: Time-Indexed Scores. The discussion below is excerpted and lightly edited from § Langevin Dynamics Preview in Mathematics for AI and Machine Learning.
The basic idea is to start from random noise and iteratively update:
where $\eta > 0$ is a step size and $\boldsymbol\epsilon_k \sim \mathcal{N}(\mathbf{0}, I)$ is Gaussian noise.
Interpretation: This is a stochastic gradient ascent on $\log p(\mathbf{x})$:
- The term $\eta\,\mathbf{s}_\theta(\mathbf{x}_k)$ moves toward higher probability regions
- The term $\sqrt{2\eta}\,\boldsymbol\epsilon_k$ injects randomness to explore the distribution
Under mild conditions, this process converges to samples from the distribution $p(\mathbf{x})$ as $k \to \infty$ and $\eta \to 0$.
What this drawing shows
What you see. Shows score fields $s_t(\mathbf{x})$ at increasing noise levels $t{=}0, t{=}1, \ldots, T$ in a diffusion process.
In the mind map. Chapter 17 — Diffusion Preview: Time-Indexed Scores. See From the book above for definitions, figures, and worked examples.
Where to read next
Read the full definitions, figures, and worked examples in Chapter 17: Score Function and Energy-Based Models — see the mind-map node Diffusion Preview: Time-Indexed Scores.