Chapter 18: Langevin Dynamics and Sampling. In the chapter mind map this icon labels Diffusion Preview: Continuous-Time SDEs. The discussion below is excerpted and lightly edited from § Derivation from Continuous-Time Limit in Mathematics for AI and Machine Learning.

The discrete update can be derived by discretizing the continuous-time Langevin SDE:

Using the Euler-Maruyama discretization (see Chapter the referenced section) with time step $\eta$:

where $\boldsymbol\epsilon \sim \mathcal{N}(\mathbf{0}, I)$ and $\sqrt{\eta} d\mathbf{W}_t \approx \sqrt{\eta} \boldsymbol\epsilon$. Setting $\sqrt{2\eta} = \sqrt{2} \sqrt{\eta}$ gives the discrete update formula.

The factor $\sqrt{2}$ comes from the requirement that the stationary distribution of the continuous process matches the target distribution $p(\mathbf{x})$. This is a consequence of the fluctuation-dissipation theorem from statistical physics.