Finite-sample optimality compares procedures without taking \(n\to\infty\). Strong universal optima are rare, so classical theory obtains them by restricting the comparison class: unbiased estimators, level-\(\alpha\) tests, invariant procedures, or one-sided alternatives in monotone-likelihood-ratio families. Admissibility and minimaxity over unrestricted risk functions belong to statistical decision theory.
Minimum-variance unbiased estimation
Definition. An estimator \(U\) is unbiased for \(g(\theta)\) if \(\mathbb E_\theta U=g(\theta)\) for every \(\theta\). It is uniformly minimum-variance unbiased if, for every unbiased \(V\) with finite variance, \[ \operatorname{Var}_\theta(U) \le\operatorname{Var}_\theta(V) \qquad\text{for every }\theta. \]
Lehmann-Scheffe theorem. If \(T\) is complete and sufficient and \(U\) is unbiased, then \[ U^\star=\mathbb E(U\mid T) \] is the unique UMVU estimator of \(\mathbb E_\theta U\).
Proof. Sufficiency makes \(U^\star\) a parameter-free function of \(T\), and iterated expectation preserves unbiasedness. Rao-Blackwell gives \[ \operatorname{Var}_\theta(U^\star) \le\operatorname{Var}_\theta(U). \] Apply the same conditioning to every competing unbiased estimator. If \(V_1(T)\) and \(V_2(T)\) are both unbiased for the same target, then \(\mathbb E_\theta(V_1-V_2)=0\) for all \(\theta\); completeness forces equality almost surely. Thus every Rao-Blackwellized competitor equals \(U^\star\).
Example (Poisson functional). If \(X_1,\ldots,X_n\) are i.i.d. \(\operatorname{Poisson}(\lambda)\), then \(T=\sum_iX_i\sim\operatorname{Poisson}(n\lambda)\) is complete sufficient. Since \[ \mathbb E\{T(T-1)\}=n^2\lambda^2, \] the estimator \(T(T-1)/n^2\) is the unique UMVU estimator of \(\lambda^2\).
Cramer-Rao inequality and equality
Multivariate information inequality. Let \(U\in\mathbb R^k\) be unbiased for \(g(\theta)\), and suppose the regular score \(s_\theta\in\mathbb R^d\) has nonsingular information \(I(\theta)\). Then \[ \operatorname{Cov}_\theta(U) \succeq Dg(\theta)I(\theta)^{-1}Dg(\theta)^\top. \]
Proof. Differentiating \(\mathbb E_\theta U=g(\theta)\) gives \[ \operatorname{Cov}_\theta(U,s_\theta)=Dg(\theta). \] Put \[ R=U-g(\theta)-Dg(\theta)I^{-1}s_\theta. \] Expanding the positive semidefinite matrix \(\mathbb E(RR^\top)\) and using \(\mathbb E(s_\theta s_\theta^\top)=I\) yields \[ \operatorname{Cov}(U)-Dg\,I^{-1}Dg^\top\succeq0. \]
Equality condition. Equality holds exactly when \[ U-g(\theta)=Dg(\theta)I(\theta)^{-1}s_\theta(X) \quad P_\theta\text{-almost surely}. \] Thus exact attainment requires the estimation error to be a linear function of the score at every parameter value, a stringent structural condition.
Monotone likelihood ratios
Definition. A family \(\{P_\theta:\theta\in\mathbb R\}\) has monotone likelihood ratio in \(T\) if, whenever \(\theta_1>\theta_0\), \[ \frac{p_{\theta_1}(x)}{p_{\theta_0}(x)} \] is a nondecreasing function of \(T(x)\).
Karlin-Rubin theorem. In a monotone-likelihood-ratio family, a test \[ \phi^\star=1_{\{T>c\}}+\gamma1_{\{T=c\}}, \qquad \mathbb E_{\theta_0}\phi^\star=\alpha, \] is uniformly most powerful of level \(\alpha\) for \[ H_0:\theta\le\theta_0 \quad\text{against}\quad H_1:\theta>\theta_0. \]
Proof. For every fixed \(\theta_1>\theta_0\), monotone likelihood ratio makes the Neyman-Pearson rejection region for \(\theta_0\) versus \(\theta_1\) an upper set in \(T\). The same critical value \(c\), determined by size at \(\theta_0\), is therefore most powerful against every \(\theta_1>\theta_0\). Moreover, stochastic monotonicity of \(T\) implies \[ \sup_{\theta\le\theta_0}\mathbb E_\theta\phi^\star =\mathbb E_{\theta_0}\phi^\star=\alpha. \] Hence the test is level \(\alpha\) for the composite null and simultaneously most powerful at every alternative.
Example (binomial one-sided test). For \(X\sim\operatorname{Binomial}(n,p)\), \[ \frac{P_{p_1}(X=x)}{P_{p_0}(X=x)} =\left(\frac{p_1(1-p_0)}{p_0(1-p_1)}\right)^x \left(\frac{1-p_1}{1-p_0}\right)^n, \] which increases in \(x\) when \(p_1>p_0\). Therefore upper-tail tests are UMP for \(p\le p_0\) against \(p>p_0\), with boundary randomization if exact size is required.
Why two-sided UMP tests usually fail
For testing \(\theta=\theta_0\) against \(\theta\ne\theta_0\), the most powerful test against \(\theta_1>\theta_0\) rejects in the upper tail, while the most powerful test against \(\theta_1<\theta_0\) rejects in the lower tail. No single level-\(\alpha\) allocation can usually be most powerful in both directions. Imposing unbiasedness leads to the weaker UMPU criterion.
Definition. A level-\(\alpha\) test of \(H_0:\theta\in\Theta_0\) is unbiased if \[ \mathbb E_\theta\phi\ge\alpha \quad\text{for every }\theta\in\Theta_1, \] when its null rejection probability is exactly \(\alpha\). It is UMPU if it maximizes power at each alternative among all unbiased level-\(\alpha\) tests.
In regular one-parameter exponential families, conditioning on a complete sufficient statistic for nuisance parameters and choosing two critical tails to satisfy both a size equation and a zero derivative of power at the null often yields a UMPU test. The classical two-sided \(t\)-test and equal-tail exact tests fit this structure.
Conditioning away nuisance parameters
Conditional exactness. Suppose \(S\) is sufficient for a nuisance parameter under the null and the conditional distribution of \(T\mid S\) depends only on the target null value. A conditional test satisfying \[ \mathbb E_{\theta,\eta}(\phi\mid S)=\alpha \quad\text{under }H_0 \] has unconditional size \(\alpha\) for every nuisance value \(\eta\).
Proof. By iterated expectation, \[ \mathbb E_{\theta,\eta}\phi =\mathbb E_{\theta,\eta}\{\mathbb E_{\theta,\eta}(\phi\mid S)\} =\alpha. \]
Fisher's exact test conditions on fixed margins so that the remaining hypergeometric law no longer contains the nuisance success probabilities. Such exactness is model-specific: conditioning can also discard power when the chosen statistic is not genuinely ancillary or sufficient for nuisance structure.