Bayes and minimax analysis are dual operations on the risk function. A prior averages risk and produces a lower bound on worst-case performance. A minimax rule supplies an upper bound. A least favorable prior is one for which these bounds meet.
Bayes and minimax values
Definition. For a prior \(\Pi\), \[ r(\Pi,\delta) =\int_\Theta R(\theta,\delta)\Pi(\mathrm{d}\theta), \qquad r_\Pi=\inf_\delta r(\Pi,\delta). \] The minimax value is \[ r^*=\inf_\delta\sup_{\theta\in\Theta}R(\theta,\delta). \] A least favorable prior maximizes \(r_\Pi\) over priors.
Proposition (weak duality). \[ \sup_\Pi\inf_\delta r(\Pi,\delta) \leq \inf_\delta\sup_{\theta\in\Theta}R(\theta,\delta). \]
Proof. For every \((\Pi,\delta)\), \[ r(\Pi,\delta) \leq\sup_\theta R(\theta,\delta). \] Taking the infimum over \(\delta\) on the left preserves the inequality, and then taking the supremum over \(\Pi\) gives the result. \(\square\)
A finite minimax theorem
Theorem. Suppose the parameter set and the set of nonrandomized rules are finite, and permit randomization over rules. Then \[ \sup_{\pi\in\Delta(\Theta)} \inf_{\delta}r(\pi,\delta) = \inf_{\delta} \sup_{\theta\in\Theta}R(\theta,\delta). \] Both extrema are attained.
Proof. Let \(A\) be the matrix whose entry \(A_{\theta j}\) is the risk of nonrandomized rule \(j\) at parameter \(\theta\). A randomized rule is a probability vector \(q\), and a prior is a probability vector \(\pi\). The bilinear payoff is \[ \pi^\top Aq. \] The simplex sets for \(\pi\) and \(q\) are compact and convex. The finite-dimensional von Neumann minimax theorem gives \[ \max_\pi\min_q\pi^\top Aq = \min_q\max_\pi\pi^\top Aq. \] For fixed \(q\), maximizing over the prior simplex selects the largest coordinate of \(Aq\), which is \(\sup_\theta R(\theta,q)\). Compactness gives attainment. \(\square\)
On general spaces, equality requires conditions such as compactness of an appropriate rule or risk set, convexity from randomization, and lower semicontinuity. Without them, least favorable priors may exist only as approximating sequences.
Least favorable priors
Theorem (Bayes–minimax certificate). Suppose \(\delta^*\) is Bayes under \(\Pi^*\) and \[ \sup_\theta R(\theta,\delta^*) = r(\Pi^*,\delta^*). \] Then \(\delta^*\) is minimax, \(\Pi^*\) is least favorable, and \[ r^*=r_{\Pi^*}=r(\Pi^*,\delta^*). \]
Proof. Weak duality and Bayes optimality give \[ r_{\Pi^*} \leq\sup_\Pi r_\Pi \leq r^* \leq\sup_\theta R(\theta,\delta^*). \] The first and last quantities are equal by assumption, so every inequality is equality. \(\square\)
A constant-risk Bayes rule automatically satisfies the equality. More generally, it is enough that its maximum risk occurs on the support of the prior and equals its prior average.
Normal mean with bounded parameter space
Example. Observe \(X\sim N(\theta,\sigma^2)\), \(\theta\in[-m,m]\), under squared-error loss. For the symmetric two-point prior \[ \Pi=\tfrac12\delta_{-m}+\tfrac12\delta_m, \] the posterior mean is \[ \delta_\Pi(x) = m\tanh\!\left(\frac{mx}{\sigma^2}\right). \] Its Bayes risk gives a computable lower bound on the minimax risk over \([-m,m]\). The exact least favorable prior generally has more support points as \(m/\sigma\) grows, illustrating that a convenient symmetric prior need not be least favorable.
Invariant location estimation
Example (Gaussian location). Observe \(X\sim N(\theta,\sigma^2)\), \(\theta\in\mathbb R\), under squared loss. The equivariant estimator \(\delta(X)=X\) has constant risk \[ R(\theta,\delta)=\sigma^2. \] Proper priors \(N(0,\tau^2)\) have Bayes estimator \[ \delta_\tau(X)=\frac{\tau^2}{\tau^2+\sigma^2}X \] and Bayes risk \[ r_{\Pi_\tau} =\frac{\tau^2\sigma^2}{\tau^2+\sigma^2} \uparrow\sigma^2 \quad\text{as }\tau\to\infty. \] Hence weak duality yields \(r^*\geq\sigma^2\), while \(X\) gives \(r^*\leq\sigma^2\). Therefore \(X\) is minimax. The flat prior is improper, so least favorability is realized by a sequence of increasingly diffuse proper priors.
Saddle points
Definition. A pair \((\Pi^*,\delta^*)\) is a saddle point if \[ r(\Pi,\delta^*) \leq r(\Pi^*,\delta^*) \leq r(\Pi^*,\delta) \] for every prior \(\Pi\) and rule \(\delta\).
The left inequality says \(\Pi^*\) concentrates on worst-case parameters for \(\delta^*\); the right says \(\delta^*\) is Bayes for \(\Pi^*\). A saddle point is therefore the exact game-theoretic expression of simultaneous minimaxity and least favorability.
Local asymptotic use
In asymptotic decision theory, one selects a finite prior on local parameters \(h_1,\ldots,h_k\), proves convergence of the corresponding finite experiments, and transfers the Bayes risk of the limit experiment as a lower bound. Taking richer finite priors or expanding compact local sets then approaches the local minimax value. This finite-prior method avoids assuming a global least favorable prior exists.