General contraction theorems turn posterior rate proofs into three verifications: prior mass near the truth, entropy of a sieve, and tests outside the target ball.
Prior mass condition
The posterior denominator is controlled by prior mass in KL-type neighborhoods.
Definition. A KL-variance neighborhood is \[ B_n(P_0,\varepsilon)=\{P:K(P_0,P)\le\varepsilon^2,\; V(P_0,P)\le\varepsilon^2\}, \] where \(V(P_0,P)=P_0[\log(p_0/p)-K(P_0,P)]^2\).
Proposition. If \(\Pi(B_n(P_0,\varepsilon_n))\ge \exp(-C n\varepsilon_n^2)\), then the posterior denominator is not smaller than \(\exp(-(C+c)n\varepsilon_n^2)\) with high \(P_0\)-probability.
Proof. Integrate the likelihood ratio over the KL-variance ball. The mean and variance bounds for log likelihood ratios imply, through exponential or Chebyshev-type control, that the integrated likelihood is not exponentially smaller than its prior mass on that ball.
Entropy of sieves
A sieve restricts attention to a high-prior-mass subset of manageable complexity.
Definition. The covering number \(N(\varepsilon,\mathcal F,d)\) is the smallest number of \(d\)-balls of radius \(\varepsilon\) needed to cover \(\mathcal F\). Its logarithm is metric entropy.
Proposition. If \(\log N(\varepsilon_n,\mathcal F_n,d)\lesssim n\varepsilon_n^2\), then tests can often be union-bounded over an \(\varepsilon_n\)-net of \(\mathcal F_n\).
Proof. A test for one separated ball has exponentially small type-II error. Applying the test to each net point and taking the maximum multiplies the error by the covering number. The entropy condition keeps the product exponentially small.
General contraction theorem
The theorem combines the denominator, sieve, and testing bounds.
Theorem. If the prior mass, sieve entropy, sieve complement mass, and testing conditions all hold at \(\varepsilon_n\), then the posterior contracts at rate \(\varepsilon_n\) up to a sufficiently large constant.
Proof. Decompose the posterior outside the target ball into the part inside the sieve and the sieve complement. Tests control the expected numerator on the sieve. The prior controls the sieve complement. The KL mass lower bound controls the denominator. Markov's inequality converts expected bounds into convergence in probability.
Choice of metric
The metric must match both testing and the scientific target.
Remark. Hellinger distance is natural for density testing; \(L^2\) is natural in Gaussian white noise; weaker metrics may yield faster rates but less inferential content.
Covering and packing numbers
Definition. The covering number \[ N(\varepsilon,\mathcal F,d) \] is the smallest number of \(d\)-balls of radius \(\varepsilon\) covering \(\mathcal F\). The packing number \[ D(\varepsilon,\mathcal F,d) \] is the largest cardinality of an \(\varepsilon\)-separated subset. They satisfy \[ D(2\varepsilon,\mathcal F,d) \le N(\varepsilon,\mathcal F,d) \le D(\varepsilon,\mathcal F,d). \]
Proof. Any \(\varepsilon\)-cover can contain at most one point of a \(2\varepsilon\)-packing in each ball, giving the first inequality. A maximal \(\varepsilon\)-packing is an \(\varepsilon\)-cover, because any uncovered point could be added, giving the second.
Uniform tests from entropy
Lemma. Suppose for each fixed \(P_1\) with \(d(P_1,P_0)\ge\varepsilon\), there is a test whose two errors are at most \(e^{-cn\varepsilon^2}\) uniformly over a ball \(B(P_1,\varepsilon/2)\). If \[ \log N(\varepsilon/2,\mathcal P,d) \le Cn\varepsilon^2 \] with \(C<c\), then there is one test of \(P_0\) against \[ \{P\in\mathcal P:d(P,P_0)\ge\varepsilon\} \] whose errors are exponentially small.
Proof. Cover the alternative by \(N\) local balls and let \(\phi\) be the maximum of their tests. Under \(P_0\), a union bound gives error at most \[ Ne^{-cn\varepsilon^2} \le e^{-(c-C)n\varepsilon^2}. \] Every alternative belongs to one ball, so its type-II error is bounded by that local test's error.
General posterior contraction theorem
Theorem. Let \(X^{(n)}\sim P_0^{(n)}\), and let \(d_n\) be a testing metric. Suppose \(\varepsilon_n\downarrow0\) and \(n\varepsilon_n^2\to\infty\). Assume there are sieves \(\mathcal P_n\) and constants \(C_1,C_2,C_3>0\) such that:
- local prior mass: \[ \Pi\{K(P_0,P)\le\varepsilon_n^2,\ V(P_0,P)\le\varepsilon_n^2\} \ge e^{-C_1n\varepsilon_n^2}; \]
- sieve entropy: \[ \log N(\varepsilon_n,\mathcal P_n,d_n) \le C_2n\varepsilon_n^2; \]
- prior tail: \[ \Pi(\mathcal P_n^c) \le e^{-(C_1+C_3)n\varepsilon_n^2}, \] with \(C_3\) large relative to the testing constant.
Then for sufficiently large \(M\), \[ \Pi\{P:d_n(P,P_0)>M\varepsilon_n \mid X^{(n)}\} \xrightarrow{P_0}0. \]
Proof. Partition the sieve alternative into shells \[ \mathcal S_j =\{P:jM\varepsilon_n<d_n(P,P_0) \le(j+1)M\varepsilon_n\}. \] Entropy and local testing produce tests \(\phi_{n,j}\) with errors bounded by \[ \exp\{-c j^2M^2n\varepsilon_n^2+C_2(j+1)^2n\varepsilon_n^2\}. \] Choose \(M\) so the exponent is negative uniformly. The KL mass condition gives a denominator lower bound of order \(e^{-(C_1+o(1))n\varepsilon_n^2}\). After multiplying posterior numerators by \(1-\phi_{n,j}\), change of measure bounds their expectations by the type-II errors. Sum the geometric shell series. The prior-tail condition handles \(\mathcal P_n^c\), and the aggregate type-I error vanishes.
Non-iid and experiment-specific forms
For independent nonidentically distributed data, replace KL divergence by \[ K_n(P_0,P) =\frac1n\sum_{i=1}^n E_{0i}\log\frac{p_{0i}}{p_i}, \] and use product-experiment tests. In Gaussian white noise, the likelihood metric is \(L^2\); in regression with random design, it is typically prediction norm \[ \|f-f_0\|_{L^2(P_X)}; \] in inverse problems, it is often the norm after the forward operator. Contraction in the parameter norm then requires an inverse-modulus argument.
Prior-dependent and target rates
Theorem (two-rate version). Suppose KL prior mass is available at rate \(\bar\varepsilon_n\), while sieve entropy is controlled at rate \(\varepsilon_n\), with \[ \varepsilon_n\ge\bar\varepsilon_n,\qquad n\bar\varepsilon_n^2\to\infty. \] Then posterior contraction occurs at \(\varepsilon_n\), provided the prior tail is smaller than the denominator scale \(e^{-Cn\bar\varepsilon_n^2}\).
This separates approximation/prior concentration from global complexity. The slower of the two controls the final rate.
Metric strengthening by interpolation
Proposition. Suppose posterior mass lies in a Hölder ball \(\|f\|_{C^\beta}\le R_n\), and contraction holds in \(L^2\) at rate \(\varepsilon_n\). An interpolation inequality of the form \[ \|f-f_0\|_\infty \lesssim \|f-f_0\|_{L^2}^{2\beta/(2\beta+1)} \|f-f_0\|_{C^\beta}^{1/(2\beta+1)} \] implies supremum contraction at rate \[ \varepsilon_n^{2\beta/(2\beta+1)} R_n^{1/(2\beta+1)}. \]
Proof. Apply the interpolation inequality on the high-posterior-probability sieve and substitute the two bounds. The posterior probability of leaving the sieve or the \(L^2\) ball vanishes.
Why entropy cannot be omitted
Remark. Large local prior mass alone can coexist with enormous mass spread over exponentially many alternatives. Tests must defeat the whole alternative, and entropy quantifies how many local tests are needed. Conversely, a tiny model with excellent entropy can contract slowly if the prior puts exponentially little mass near truth. Rate proofs need both local concentration and global complexity.
The covering, testing, and empirical-process machinery behind these bounds is developed in empirical processes and statistical decision theory.