The Bernstein-von Mises theorem says that, in regular finite-dimensional models, the posterior is asymptotically Gaussian and prior effects disappear at first order.
Total variation formulation
The strongest common formulation compares the whole posterior law, not only means and variances.
Definition. Let \(\hat\theta_n\) be an efficient estimator. A Bernstein-von Mises theorem states \[ \left\|\Pi\left(\sqrt n(\theta-\hat\theta_n)\in\cdot\mid X^{(n)}\right) -N(0,I_{\theta_0}^{-1})\right\|_{\mathrm{TV}}\to0 \] in \(P_{\theta_0}\)-probability.
Remark. Centering at \(\hat\theta_n\) removes the random first-order shift. Centering at \(\theta_0\) gives a Gaussian with random mean \(I_{\theta_0}^{-1}\Delta_n\).
Proof mechanism
The proof has three conceptual parts: localization, LAN, and prior flatness.
Theorem. Assume posterior localization at \(n^{-1/2}\)-scale, LAN at \(\theta_0\), nonsingular Fisher information, and a prior density continuous and positive at \(\theta_0\). Then the finite-dimensional BvM conclusion holds.
Proof. Write the posterior in local coordinates \(h=\sqrt n(\theta-\theta_0)\). Outside large compact sets, localization makes the posterior mass negligible. On compact sets, LAN and prior continuity make the posterior density uniformly close to a Gaussian kernel. Normalizing constants converge by dominated convergence on compact sets plus the tail bound, yielding total variation convergence.
Credible ellipsoids
BvM connects Bayesian credible regions with frequentist confidence regions.
Corollary. Let \(C_n=\{\theta:n(\theta-\hat\theta_n)^\top I_{\theta_0}(\theta-\hat\theta_n)\le\chi^2_{d,1-\alpha}\}\). Under BvM and efficient centering, \(C_n\) has asymptotic posterior probability \(1-\alpha\) and frequentist coverage \(1-\alpha\).
Proof. Posterior credibility follows from convergence of the local posterior to the standard chi-square radius. Frequentist coverage follows because \(\sqrt n(\hat\theta_n-\theta_0)\Rightarrow N(0,I_{\theta_0}^{-1})\).
Where BvM fails
BvM is not automatic in nonregular or infinite-dimensional problems.
Remark. Boundary parameters, weak identifiability, growing dimension, nonparametric models, and irregular functionals can preserve prior influence or produce non-Gaussian limits.
Regular finite-dimensional theorem
Theorem (Bernstein-von Mises). Let \(\theta\in\Theta\subset\mathbb R^d\), with true interior point \(\theta_0\). Suppose:
- the model is identifiable and differentiable in quadratic mean at \(\theta_0\);
- \(I_{\theta_0}\) is nonsingular;
- the prior has a density continuous and positive at \(\theta_0\);
- the posterior is consistent and localizes at \(n^{-1/2}\) rate;
- the LAN remainder is uniform on posterior-relevant local sets.
Then \[ \left\| \Pi\left( \sqrt n(\theta-\widehat\theta_n)\in\cdot \mid X^{(n)} \right) -N_d(0,I_{\theta_0}^{-1}) \right\|_{\mathrm{TV}} \xrightarrow{P_{\theta_0}}0, \] for any efficient estimator satisfying \[ \sqrt n(\widehat\theta_n-\theta_0) =I_{\theta_0}^{-1}\Delta_n+o_P(1). \]
Proof outline. Change variables to \(h=\sqrt n(\theta-\theta_0)\). On a fixed compact set, LAN and prior continuity make the unnormalized posterior density converge in probability to \[ \exp\left\{ -\frac12(h-I^{-1}\Delta_n)^\top I(h-I^{-1}\Delta_n) \right\}. \] Posterior localization makes the mass outside a growing compact set negligible. Uniform LAN controls the intermediate annulus, while Gaussian tails control the limiting kernel. Scheffé's lemma converts \(L^1\) convergence of normalized densities to total variation. Recenter using the efficient expansion of \(\widehat\theta_n\).
Consequences for posterior moments
Proposition. Total-variation BvM guarantees convergence of posterior expectations only for uniformly bounded functions. To conclude \[ E(\theta\mid X^{(n)}) =\widehat\theta_n+o_P(n^{-1/2}) \] and \[ n\operatorname{Var}(\theta\mid X^{(n)}) \xrightarrow{P}I_{\theta_0}^{-1}, \] one additionally needs uniform integrability of the local posterior first and second moments.
Proof. Total variation controls bounded truncations of \(h\) and \(hh^\top\). Uniform integrability makes the truncated-tail expectations vanish uniformly, permitting removal of the truncation. Transform back from \(h=\sqrt n(\theta-\widehat\theta_n)\).
Smooth functional delta method
Theorem. Let \(\psi:\mathbb R^d\to\mathbb R^k\) be differentiable at \(\theta_0\), with derivative matrix \(\dot\psi_{\theta_0}\). Under BvM, \[ \sqrt n\{\psi(\theta)-\psi(\widehat\theta_n)\} \mid X^{(n)} \] converges in total variation, under suitable localization, to \[ N_k\left( 0,\, \dot\psi_{\theta_0} I_{\theta_0}^{-1} \dot\psi_{\theta_0}^\top \right). \]
Proof. Uniformly on bounded local \(h\), \[ \sqrt n\{ \psi(\theta_0+h/\sqrt n)-\psi(\theta_0) \} =\dot\psi_{\theta_0}h+o(1). \] Push the local Gaussian posterior through this asymptotically linear map and use posterior localization.
Prior disappearance is first order
BvM says that any fixed prior density positive and continuous at truth disappears from the first-order local shape. It does not say priors are irrelevant:
- they determine support and can exclude truth;
- they affect \(O(n^{-1})\) bias and finite-sample tails;
- they influence marginal likelihood at \(O(1)\);
- they can dominate weakly identified or nonregular directions;
- they remain central in prediction outside observed support.
Explicit normal-location example
Example. If \[ X_i\sim N(\theta,\sigma^2), \qquad \theta\sim N(m_0,\tau_0^2), \] then \[ \theta\mid X \sim N(m_n,v_n), \] \[ v_n=\left(\frac n{\sigma^2}+\frac1{\tau_0^2}\right)^{-1}, \qquad m_n=v_n\left(\frac{n\bar X}{\sigma^2} +\frac{m_0}{\tau_0^2}\right). \] As \(n\to\infty\), \[ nv_n\to\sigma^2,\qquad \sqrt n(m_n-\bar X)\to0. \] Thus the posterior centered at the MLE \(\bar X\) has exact asymptotic variance \(\sigma^2/n\), independent of the fixed prior.
Concrete failure mechanisms
Example (uniform endpoint). If \[ X_i\sim\operatorname{Uniform}(0,\theta), \] the MLE is \(X_{(n)}\), and \[ n\frac{\theta_0-X_{(n)}}{\theta_0} \Rightarrow\operatorname{Exp}(1). \] The local rate is \(n^{-1}\), not \(n^{-1/2}\), and the likelihood has a moving support boundary. The posterior local limit is one-sided exponential-like rather than Gaussian.
Example (weak mixture identification). In an overfitted finite mixture, component labels are nonidentified and extra component weights may converge at nonstandard rates. The posterior can remain multimodal or concentrate on singular strata, so nonsingular Fisher information and ordinary LAN fail.
In high-dimensional and nonparametric problems, BvM must be restated for a specified low-dimensional functional and topology. A global Gaussian approximation to the whole posterior is usually false.