Bayesian inference becomes decision theory once a loss function is specified. The posterior distribution is then used through posterior risk, not only through point summaries.
Posterior risk
A Bayes action minimizes the conditional expected loss after seeing the data.
Definition. Let \(a\in\mathcal A\) be an action and \(L(\theta,a)\) a loss. The posterior risk is \[ \rho(a\mid X)=\int_\Theta L(\theta,a)\,\Pi(\mathrm{d}\theta\mid X). \] A Bayes action is any measurable minimizer \(\delta(X)\in\arg\min_a\rho(a\mid X)\).
Theorem. If a measurable posterior-risk minimizer exists, then it minimizes the global Bayes risk \(r(\delta)=\int R(\theta,\delta)\Pi(\mathrm{d}\theta)\).
Proof. By Fubini and the definition of posterior, \[ r(\delta)=\mathbb E_\Pi L(\theta,\delta(X)) =\mathbb E_\Pi\{\rho(\delta(X)\mid X)\}. \] If \(\delta(X)\) minimizes the integrand for each \(X\), then the expectation is no larger than for any other decision rule.
Quadratic and absolute losses
Classical posterior summaries arise as Bayes actions for different losses.
Proposition. Under squared-error loss \(L(\theta,a)=\|\theta-a\|^2\), the Bayes action is \(a=\mathbb E(\theta\mid X)\), whenever the posterior second moment is finite. Under one-dimensional absolute loss, any posterior median is a Bayes action.
Proof. For squared loss, \[ \mathbb E\|\theta-a\|^2=\mathbb E\|\theta-\bar\theta\|^2+\|a-\bar\theta\|^2 \] with \(\bar\theta=\mathbb E(\theta\mid X)\). For absolute loss, a subgradient of \(a\mapsto\mathbb E|\theta-a|\) is \(\Pi(\theta\le a\mid X)-\Pi(\theta\ge a\mid X)\), so zero belongs to the subgradient exactly at a median.
Credible decisions
Credible sets become decision procedures only after one states what loss or coverage target they are intended to serve.
Definition. A \(1-\alpha\) credible set is a measurable set \(C(X)\) such that \(\Pi(C(X)\mid X)\ge 1-\alpha\). A highest posterior density set has the form \(\{\theta:p(\theta\mid X)\ge c_X\}\) when a posterior density exists.
Remark. Posterior credibility is a conditional probability statement under the Bayesian joint law. Frequentist coverage is a different repeated-sampling property and requires separate analysis.
Randomized actions
For convex losses, randomization is usually unnecessary; for nonconvex losses it can be part of the decision space.
Proposition. If \(L(\theta,a)\) is convex in \(a\), then replacing a randomized action by its conditional mean cannot increase posterior risk.
Proof. Condition on the data and on the posterior draw of \(\theta\). Jensen's inequality gives \(L(\theta,\mathbb E A)\le \mathbb E\{L(\theta,A)\}\). Integrating over the posterior proves the claim.
Existence of Bayes actions
Theorem. Let the action space \(\mathcal A\) be compact, and suppose \(a\mapsto L(\theta,a)\) is lower semicontinuous for every \(\theta\) and bounded below by an integrable function. Then the posterior risk \[ \rho_x(a)=\int L(\theta,a)\Pi(\mathrm{d}\theta\mid x) \] is lower semicontinuous and attains its minimum on \(\mathcal A\).
Proof. If \(a_n\to a\), Fatou's lemma gives \[ \rho_x(a) \le\liminf_n\rho_x(a_n), \] after subtracting the integrable lower bound. Thus \(\rho_x\) is lower semicontinuous. A lower semicontinuous function on a compact space attains its minimum.
Posterior quantiles under asymmetric loss
Theorem. For \(0<\tau<1\), define check loss \[ L_\tau(\theta,a) =(\theta-a)\{\tau-1(\theta<a)\}. \] Every posterior \(\tau\)-quantile minimizes posterior expected loss.
Proof. The subdifferential of the posterior risk is \[ \partial\rho_x(a) =\big[ \Pi(\theta<a\mid x)-\tau,\, \Pi(\theta\le a\mid x)-\tau \big]. \] Zero belongs to this interval exactly when \[ \Pi(\theta<a\mid x)\le\tau\le \Pi(\theta\le a\mid x), \] the defining condition for a posterior \(\tau\)-quantile.
Example (inventory asymmetry). If underestimation costs \(c_u\) per unit and overestimation costs \(c_o\), then \[ L(\theta,a) =c_u(\theta-a)_+ +c_o(a-\theta)_+. \] The optimal action is the posterior quantile \[ \tau=\frac{c_u}{c_u+c_o}. \] The posterior median is optimal only when the two costs are equal.
Testing as a Bayesian action
Proposition. For hypotheses \(H_0:\theta\in\Theta_0\) and \(H_1:\theta\in\Theta_1\), suppose rejecting \(H_0\) when it is true costs \(c_{10}\), while accepting \(H_0\) under \(H_1\) costs \(c_{01}\). The Bayes rule rejects exactly when \[ \frac{\Pi(\Theta_1\mid x)} {\Pi(\Theta_0\mid x)} >\frac{c_{10}}{c_{01}}. \]
Proof. The posterior risk of rejecting is \(c_{10}\Pi(\Theta_0\mid x)\); that of accepting is \(c_{01}\Pi(\Theta_1\mid x)\). Choose the smaller and rearrange.
Posterior probability is therefore not itself a decision threshold. The threshold encodes losses. Reporting a posterior probability while concealing the implied cost ratio leaves the action rule incomplete.
Bayes risk and frequentist risk
Definition. For a decision rule \(\delta(X)\), \[ R(\theta,\delta) =E_\theta L\{\theta,\delta(X)\} \] is its frequentist risk, and \[ r(\Pi,\delta) =\int R(\theta,\delta)\Pi(\mathrm{d}\theta) \] is its Bayes risk.
Theorem. A rule minimizing posterior risk for almost every \(x\) minimizes Bayes risk.
Proof. Fubini's theorem gives \[ r(\Pi,\delta) =\int \left[ \int L\{\theta,\delta(x)\} \Pi(\mathrm{d}\theta\mid x) \right]m(\mathrm{d}x). \] Pointwise minimization of the bracket minimizes its integral.
Posterior regret
Definition. If \(a_x^\star\) is a Bayes action, the posterior regret of \(a\) is \[ \operatorname{Reg}_x(a) =\rho_x(a)-\rho_x(a_x^\star)\ge0. \]
Proposition. Under squared loss, \[ \operatorname{Reg}_x(a) =\{a-E(\theta\mid x)\}^2. \]
Proof. The bias-variance identity gives \[ E\{(\theta-a)^2\mid x\} =\operatorname{Var}(\theta\mid x) +\{a-E(\theta\mid x)\}^2. \] Subtract the minimum posterior risk, which is the posterior variance.
Decision-theoretic comparisons, minimaxity, admissibility, and least favorable priors are developed in statistical decision theory.