Laplace approximation

Laplace's method approximates the integral of a function $\int f(\mathbf{w}) d\mathbf{w}$ by fitting a Gaussian at the maximum $\widehat{\mathbf{w}}$ of $f(\mathbf{w})$, and computing the volume under the Gaussian. The covariance of the fitted Gaussian is determined by the Hessian matrix of $\log f(\mathbf{w})$ at the maximum point $\widehat{\mathbf{w}}$ [40].

The same name is also used for the method of approximating the posterior distribution with a Gaussian centered at the maximum a posteriori estimate. This can be justified by the fact that under certain regularity conditions, the posterior distribution approaches Gaussian distribution as the number of samples grows [16].

Despite using a full distribution to approximate the posterior, Laplace's method still suffers from most of the problems of MAP estimation. Estimating the variances at the end does not help if the procedure has already lead to an area of low probability mass.