Consistency asks whether posterior mass reaches every fixed neighborhood of the truth. Contraction asks how fast the posterior neighborhood can shrink.
KL support and tests
The basic posterior consistency proof has a denominator lower bound and a numerator upper bound.
Definition. A prior has the KL property at \(P_0\) if for every \(\varepsilon>0\), \[ \Pi\{P:K(P_0,P)<\varepsilon\}>0. \]
Theorem. Schwartz's theorem: KL support at \(P_0\) plus uniformly exponentially consistent tests for the complement of a neighborhood \(U\) imply \(\Pi(U^c\mid X^{(n)})\to0\) under \(P_0\).
Proof. The posterior bad mass is a numerator divided by a denominator. KL support gives a lower bound on the denominator of order \(\exp(-n\varepsilon)\) with high probability. The tests split the numerator into type-I and type-II terms, both exponentially small on \(U^c\). Choosing constants appropriately forces the ratio to vanish.
Doob consistency
Doob's theorem is powerful but answers a different question from fixed-truth frequentist consistency.
Theorem. Under identifiability and standard measurability conditions, the posterior is consistent for \(\Pi\)-almost every parameter value.
Proof. Let \(\Theta\) be the random parameter drawn from \(\Pi\), and let \(X_1,X_2,\ldots\) be drawn conditionally from \(P_\Theta^\infty\). Identifiability implies that the tail sigma-field generated by the infinite observation sequence determines \(\Theta\) almost surely. The posterior is the conditional law of \(\Theta\) given the first \(n\) observations; by martingale convergence, these conditional laws converge to the conditional law given the infinite sequence, which is the point mass at the true \(\Theta\) for \(\Pi\)-almost every parameter.
Remark. The exceptional set may depend on the prior and can contain scientifically important truths. Schwartz-type theorems are designed to verify consistency at a fixed \(P_0\).
Contraction rates
A rate \(\varepsilon_n\) is meaningful when posterior mass outside \(M\varepsilon_n\)-balls vanishes.
Definition. The posterior contracts at rate \(\varepsilon_n\) around \(P_0\) for metric \(d\) if, for sufficiently large \(M\), \[ \Pi(P:d(P,P_0)>M\varepsilon_n\mid X^{(n)})\to0 \] in \(P_0\)-probability.
Testing scale
The rate is governed by the smallest scale at which tests, entropy, and prior mass all work.
Remark. The usual inequality \(n\varepsilon_n^2\) appears because testing errors for Hellinger-separated distributions are exponentially small in \(n d^2\), while entropy and prior-mass terms must fit under that exponential budget.
Contraction as a random probability statement
Definition. A posterior contracts around \(P_0\) at rate \(\varepsilon_n\) in metric \(d\) if, for sufficiently large \(M\), \[ \Pi\{P:d(P,P_0)>M\varepsilon_n\mid X^{(n)}\} \xrightarrow{P_0}0. \] A stronger mean statement is \[ E_0\Pi\{d(P,P_0)>M\varepsilon_n\mid X^{(n)}\} \to0. \]
The constant \(M\) distinguishes a rate statement from exact asymptotic shape. Contraction says posterior mass enters a ball of the right order; it does not say the rescaled posterior is Gaussian, centered without bias, or yields calibrated credible sets.
From contraction to posterior risk
Proposition. Suppose \(d(P,P_0)\le D\) and contraction holds at rate \(\varepsilon_n\). Then \[ E^\Pi\{d(P,P_0)\mid X^{(n)}\} =O_{P_0}(\varepsilon_n) \] if posterior tails satisfy a geometric shell bound \[ \Pi\{d(P,P_0)>jM\varepsilon_n\mid X^{(n)}\} \le Ce^{-cj^2n\varepsilon_n^2} \] with high \(P_0\)-probability.
Proof. Use the tail integral identity \[ E^\Pi d=\int_0^D\Pi(d>t\mid X)\,\mathrm{d}t. \] The interval up to \(M\varepsilon_n\) contributes at most that amount. Divide the remainder into shells of width \(M\varepsilon_n\) and sum their exponentially decaying posterior probabilities.
Kullback-Leibler prior mass
Lemma (denominator lower bound). If \[ \Pi\{P:K(P_0,P)\le\varepsilon_n^2,\ V(P_0,P)\le\varepsilon_n^2\} \ge e^{-Cn\varepsilon_n^2}, \] then for every \(c>C+1\), the posterior denominator \[ D_n=\int \prod_{i=1}^n\frac{p(X_i)}{p_0(X_i)} \Pi(\mathrm{d}P) \] satisfies \[ P_0\{D_n<e^{-cn\varepsilon_n^2}\}\to0 \] when \(n\varepsilon_n^2\to\infty\).
Proof outline. Restrict the integral to the KL neighborhood and normalize the prior there. Jensen's inequality lower-bounds its log by the prior average log likelihood ratio. Its \(P_0\)-mean is at least \(-n\varepsilon_n^2\), and the variance condition controls deviations at scale \(n\varepsilon_n^2\). Multiply by the neighborhood's prior mass.
Testing separated alternatives
Theorem (Hellinger testing). For any two densities \(p,q\), there is a test \(\phi_n\) such that \[ P^n\phi_n+Q^n(1-\phi_n) \le \left(\int\sqrt{pq}\,\mathrm{d}\mu\right)^n =\{1-h^2(P,Q)\}^n \le e^{-nh^2(P,Q)}. \]
Proof. The Neyman-Pearson test has total error \[ \int\min(p^{(n)},q^{(n)})\,\mathrm{d}\mu^n. \] Since \(\min(a,b)\le\sqrt{ab}\), this is bounded by the product Hellinger affinity \[ \left(\int\sqrt{pq}\,\mathrm{d}\mu\right)^n. \] Use \(1-x\le e^{-x}\).
Metric conversion
Proposition. For probability densities, \[ h^2(P,Q) \le\|P-Q\|_{\mathrm{TV}} \le\sqrt2\,h(P,Q), \] under the convention \[ h^2(P,Q)=\frac12\int(\sqrt p-\sqrt q)^2. \]
Proof. Since \[ |p-q| =|\sqrt p-\sqrt q|(\sqrt p+\sqrt q), \] Cauchy-Schwarz gives the upper TV bound. For the lower bound, pointwise \[ (\sqrt p-\sqrt q)^2 \le|p-q|, \] and integrate with the normalization factors.
Hellinger contraction therefore implies total-variation contraction at the same order up to constants only in the upper direction shown; stronger norms such as supremum distance require additional regularity and interpolation.
Rate lower bounds and prior mismatch
Remark. A posterior cannot contract uniformly faster than the minimax estimation rate over a parameter class without paying elsewhere, because a posterior point summary would then produce a faster estimator. It can also contract slower than minimax when its prior regularity mismatches truth, its local small-ball probability is too small, or its hyperprior undersupplies the oracle scale.
Contraction does not imply a center
Example. A posterior could place half its mass at \(f_0+\varepsilon_nv\) and half at \(f_0-\varepsilon_nv\). It contracts at rate \(\varepsilon_n\), but its rescaled shape is bimodal. Alternatively, it could concentrate near \(f_0+b_n\) with \(\|b_n\|\asymp\varepsilon_n\), yielding a biased center. Contraction alone distinguishes neither case.
Three ingredients to check
For a concrete prior and experiment, a rate proof should identify:
- approximation and prior concentration: enough mass near a model approximation to \(P_0\);
- complexity control: a sieve with entropy within the testing budget and exponentially small prior complement;
- testing: exponentially consistent tests in the metric of interest or a metric that controls it.
The next page states the general theorem assembling these ingredients.