Robust Bayesian modeling changes the likelihood or prior so that inference is less brittle under outliers, missingness, or data-collection mechanisms.
Heavy-tailed observation models
A common robust move is to replace Gaussian errors with heavier-tailed errors.
Definition. The Student \(t_\nu\) error model can be represented as \[ Y_i\mid\theta,\lambda_i\sim N(\theta,\sigma^2/\lambda_i),\qquad \lambda_i\sim\operatorname{Gamma}(\nu/2,\nu/2). \]
Proposition. Integrating out \(\lambda_i\) gives the Student \(t_\nu\) density for \(Y_i-\theta\).
Proof. The normal density contributes \(\lambda_i^{1/2}\exp\{-\lambda_i(y_i-\theta)^2/(2\sigma^2)\}\). Multiplication by the gamma density gives a gamma kernel in \(\lambda_i\). Integrating the gamma kernel yields a power term of the form \((1+(y_i-\theta)^2/(\nu\sigma^2))^{-(\nu+1)/2}\).
Missing data mechanisms
Bayesian missing-data analysis models the unobserved data jointly with the observed data.
Definition. Let \(Y=(Y_{\mathrm{obs}},Y_{\mathrm{mis}})\) and let \(R\) be the missingness pattern. The mechanism is missing at random if \[ P(R\mid Y_{\mathrm{obs}},Y_{\mathrm{mis}},\psi)=P(R\mid Y_{\mathrm{obs}},\psi). \]
Proposition. Under missing at random and parameter distinctness between \(\theta\) and \(\psi\), likelihood-based inference for \(\theta\) may ignore the missingness mechanism.
Proof. The observed likelihood factors into a term involving \(\theta\) through \(\int p_\theta(y_{\mathrm{obs}},y_{\mathrm{mis}})\,\mathrm{d}y_{\mathrm{mis}}\) and a term involving \(\psi\) through the missingness model. If the parameter spaces are distinct and the prior factorizes, the posterior for \(\theta\) is proportional to the first factor times its prior.
Contamination neighborhoods
Robustness can be formalized by asking how posterior summaries change under local perturbations of the model.
Remark. Mixture contamination, scale mixtures, and bounded-influence likelihoods all reduce sensitivity to observations that would dominate a thin-tailed Gaussian likelihood.
Data-collection design
The sampling design can be included in the joint law rather than treated as an afterthought.
Proposition. If the design is ignorable given observed covariates, then conditioning on the design variables can be sufficient for posterior inference about the response model.
Proof. This is the same factorization logic as missing-data ignorability: once the design likelihood involves no unknown parameter shared with the response model, it contributes only a multiplicative factor independent of the target parameter.
Student models as scale mixtures
Theorem. If \[ Y\mid\mu,\sigma^2,\lambda \sim N(\mu,\sigma^2/\lambda), \qquad \lambda\sim\operatorname{Gamma}(\nu/2,\nu/2), \] then \[ \frac{Y-\mu}{\sigma}\sim t_\nu. \]
Proof. Integrating \(\lambda\) gives a density proportional to \[ \int_0^\infty \lambda^{1/2} e^{-\lambda(y-\mu)^2/(2\sigma^2)} \lambda^{\nu/2-1}e^{-\nu\lambda/2} \,\mathrm{d}\lambda. \] The gamma integral equals a constant times \[ \left\{1+\frac{(y-\mu)^2}{\nu\sigma^2}\right\}^{-(\nu+1)/2}, \] the Student density.
The latent precision \(\lambda_i\) becomes small for an outlying residual, reducing that observation's conditional influence. Robustness is achieved by modeling heavy tails, not by declaring inconvenient observations irrelevant.
Contamination mixtures
Definition. A two-component contamination model is \[ Y_i\mid z_i,\theta \sim(1-\epsilon)p_\theta(y_i) +\epsilon q_\psi(y_i), \qquad z_i\sim\operatorname{Bernoulli}(\epsilon), \] where \(z_i=1\) marks the broad contamination component.
Proposition. Conditional on parameters, \[ P(z_i=1\mid y_i) =\frac{\epsilon q_\psi(y_i)} {(1-\epsilon)p_\theta(y_i)+\epsilon q_\psi(y_i)}. \]
Proof. Apply Bayes' formula to the two mixture components.
Posterior outlier probabilities depend on the specified alternative \(q_\psi\), the contamination rate prior, and separation between components. They are model-based classifications, not objective labels.
Observed-data factorization
Definition. Write complete data as \(Y=(Y_{\mathrm{obs}},Y_{\mathrm{mis}})\) and missingness pattern \(R\). A selection model factors \[ p(Y,R\mid\theta,\psi) =p(Y\mid\theta)p(R\mid Y,\psi). \] Missingness is:
- MCAR if \(p(R\mid Y,\psi)=p(R\mid\psi)\);
- MAR if \(p(R\mid Y,\psi)=p(R\mid Y_{\mathrm{obs}},\psi)\);
- MNAR when dependence on \(Y_{\mathrm{mis}}\) remains.
Theorem (Bayesian ignorability). Suppose missingness is MAR, the parameters \(\theta\) and \(\psi\) are distinct, and the prior factors as \(\pi(\theta,\psi)=\pi(\theta)\pi(\psi)\). Then \[ \pi(\theta\mid Y_{\mathrm{obs}},R) \propto \pi(\theta) \int p(Y_{\mathrm{obs}},Y_{\mathrm{mis}}\mid\theta) \,\mathrm{d}Y_{\mathrm{mis}}, \] so the missingness model is ignorable for posterior inference on \(\theta\).
Proof. The observed likelihood is \[ \int p(Y\mid\theta) p(R\mid Y_{\mathrm{obs}},\psi) \pi(\theta)\pi(\psi) \,\mathrm{d}Y_{\mathrm{mis}}\,\mathrm{d}\psi. \] The missingness factor does not depend on \(Y_{\mathrm{mis}}\) or \(\theta\), so the integral factors into a \(\theta\)-free term times the displayed observed-data likelihood.
Data augmentation
Proposition. Alternating draws \[ Y_{\mathrm{mis}}^{(s+1)} \sim p(Y_{\mathrm{mis}}\mid Y_{\mathrm{obs}},R,\theta^{(s)}), \] \[ \theta^{(s+1)} \sim p(\theta\mid Y_{\mathrm{obs}},Y_{\mathrm{mis}}^{(s+1)}) \] define a Gibbs chain with invariant distribution \[ p(\theta,Y_{\mathrm{mis}} \mid Y_{\mathrm{obs}},R). \]
Proof. The two updates are the full conditionals of the augmented posterior. Invariance follows from the Gibbs-kernel argument.
Nonidentifiability under MNAR
Proposition. Without restrictions, the observed law of \((Y_{\mathrm{obs}},R)\) does not identify both the full-data distribution and an MNAR selection mechanism.
Proof. In the simplest binary case, observed data identify \[ P(R=1,Y=1),\quad P(R=1,Y=0),\quad P(R=0). \] They do not identify the split \[ P(R=0,Y=1)+P(R=0,Y=0)=P(R=0). \] Any split can be paired with a corresponding outcome distribution and response mechanism to produce the same observed law.
MNAR analysis therefore requires sensitivity parameters, exclusion restrictions, validation data, or a scientifically justified joint model. Posterior concentration around one extrapolation does not create identification.
Bayesian design and stopping
A full Bayesian design averages utilities over the prior predictive: \[ U(d) =E_{Y\sim m_d} \left[ \max_a E\{u(\theta,a,d)\mid Y,d\} \right] -c(d). \] Adaptive designs update after interim data and choose the next design action. If the stopping rule depends only on observed data and has no parameter-dependent factor beyond the likelihood, posterior inference conditional on the realized data obeys the likelihood principle. Operating characteristics such as type-I error, power, and sample-size distribution still depend on the stopping rule and remain essential for regulatory or repeated-use evaluation.