A sufficient statistic is a lossless compression of an experiment. The ordinary definition through conditional distributions, the factorization theorem, and the decision-theoretic statement that no risk is lost are all expressions of one kernel equivalence.
A statistic as deterministic post-processing
Definition. A measurable statistic \(T:\mathcal X\to\mathcal Y\) induces the reduced experiment \[ T\mathcal E = (\mathcal Y,\mathcal A_{\mathcal Y}, \{Q_\theta=P_\theta T^{-1}:\theta\in\Theta\}). \] The deterministic kernel \[ K_T(x,B)=1\{T(x)\in B\} \] maps \(\mathcal E\) to \(T\mathcal E\).
Thus the full observation is always at least as informative as its statistic. Sufficiency asks whether a parameter-free kernel can reconstruct an observation with the correct law from the statistic.
Theorem (kernel characterization). Suppose the spaces are standard Borel. The statistic \(T\) is sufficient for \(\{P_\theta\}\) if and only if a Markov kernel \(K:\mathcal Y\rightsquigarrow\mathcal X\), independent of \(\theta\), exists such that \[ Q_\theta K=P_\theta \quad\text{for every }\theta. \] Consequently, \[ \Delta(\mathcal E,T\mathcal E)=0. \]
Proof. If \(T\) is sufficient, a version of the conditional law \[ K(t,A)=P_\theta(X\in A\mid T=t) \] can be chosen independently of \(\theta\). Disintegration gives \[ P_\theta(A)=\int K(t,A)Q_\theta(\mathrm{d}t). \] Conversely, such a reconstruction kernel supplies a parameter-free conditional simulation of \(X\) from \(T\), which is the conditional-distribution definition of sufficiency. The deterministic statistic kernel sends \(\mathcal E\) to \(T\mathcal E\), and \(K\) sends it back, so both deficiencies are zero. \(\square\)
Equality of attainable risks
Corollary. If \(T\) is sufficient, then for every action space and loss, every rule \(\delta\) based on \(X\) has a rule \(\rho\) based on \(T\) with \[ R(\theta,\rho)=R(\theta,\delta) \quad\text{for all }\theta. \]
Proof. Given \(T=t\), draw \(X'\sim K(t,\cdot)\), then apply \(\delta(X',\cdot)\). The composed rule is \[ \rho(t,C)=\int_{\mathcal X}\delta(x,C)K(t,\mathrm{d}x). \] Under \(Q_\theta\), the simulated \(X'\) has law \(Q_\theta K=P_\theta\), so all action distributions and risks match. \(\square\)
The Fisher–Neyman factorization theorem
Theorem. Suppose \(P_\theta\ll\mu\) with densities \(p_\theta\). If \[ p_\theta(x)=g_\theta(T(x))h(x) \] for nonnegative measurable \(g_\theta\) and \(h\), then \(T\) is sufficient. Under standard regularity and a common dominating measure, sufficiency also implies such a factorization up to \(\mu\)-null sets.
Proof of sufficiency. Define the measure \(\nu(\mathrm{d}x)=h(x)\mu(\mathrm{d}x)\). Conditional on \(T=t\), the factor \(g_\theta(t)\) is constant and cancels from normalized conditional probabilities. Hence the conditional law of \(X\) given \(T=t\) under \(P_\theta\) equals a version of the conditional law under \(\nu\), independent of \(\theta\). The kernel characterization applies. \(\square\)
Example (exponential family). For \[ p_\eta(x_1,\ldots,x_n) = \exp\left\{ \eta^\top\sum_{i=1}^nT(x_i)-nA(\eta) \right\} \prod_{i=1}^nh(x_i), \] the sum \(\sum_iT(X_i)\) is sufficient. The original \(n\)-sample experiment and the distribution of this finite-dimensional statistic are exactly equivalent.
Rao–Blackwellization as decision reduction
Theorem (Rao–Blackwell). Let \(T\) be sufficient, let the action space be convex in a vector space, and suppose \(a\mapsto L(\theta,a)\) is convex. For any nonrandomized estimator \(\delta(X)\) with integrable action, define \[ \delta_T(T)=E\{\delta(X)\mid T\}. \] Then \[ R(\theta,\delta_T)\leq R(\theta,\delta) \quad\text{for every }\theta. \]
Proof. Sufficiency makes the conditional distribution of \(X\) given \(T\), and hence \(\delta_T\), independent of the unknown parameter. Conditional Jensen inequality gives \[ L\{\theta,E_\theta(\delta\mid T)\} \leq E_\theta\{L(\theta,\delta)\mid T\}. \] Take \(P_\theta\)-expectations. \(\square\)
For strictly convex loss, equality requires the original estimator to be essentially a function of \(T\). Thus a sufficient statistic not only retains all randomized risks; it often identifies a canonical improved nonrandomized estimator.
Minimal sufficiency
Definition. A sufficient statistic \(T\) is minimal sufficient if, for every sufficient statistic \(S\), there is a measurable map \(f\) such that \[ T=f(S) \quad P_\theta\text{-almost surely for every }\theta. \]
Theorem (likelihood-ratio criterion). In a dominated model, if a statistic \(T\) satisfies \[ T(x)=T(y) \quad\Longleftrightarrow\quad \frac{p_\theta(x)}{p_\theta(y)} \text{ is independent of }\theta \] whenever the ratios are defined, then \(T\) is minimal sufficient.
Minimal sufficiency concerns exact compression within one experiment. Le Cam equivalence extends the same idea to approximate compression between different sample spaces and asymptotic models.