Prior construction balances invariance, information, support, and posterior propriety. Weak information does not mean no information; it means deliberately controlled information.
Jeffreys prior and reparametrization
Jeffreys prior is built from the local curvature of the statistical model.
Definition. For a regular one-parameter model with Fisher information \[ I(\theta)=\mathbb E_\theta\left[\left(\frac{\partial}{\partial\theta}\log p_\theta(X)\right)^2\right], \] Jeffreys prior is \(\pi_J(\theta)\propto I(\theta)^{1/2}\). In dimension \(d\), \(\pi_J(\theta)\propto\det I(\theta)^{1/2}\).
Proposition. Jeffreys prior is invariant under smooth one-to-one reparametrization.
Proof. If \(\phi=g(\theta)\), then \(I_\phi(\phi)=I_\theta(\theta)(\mathrm{d}\theta/\mathrm{d}\phi)^2\) in one dimension, and \(I_\phi=J^\top I_\theta J\) in several dimensions. Taking square roots of determinants gives the Jacobian factor required for density transformation.
Improper priors and posterior propriety
Improper priors can be useful notation, but the posterior must still normalize.
Definition. An improper prior is a sigma-finite measure \(\Pi\) with \(\Pi(\Theta)=\infty\). It defines a posterior only when \[ 0<\int p_\theta(X)\,\Pi(\mathrm{d}\theta)<\infty \] for the observed data, or almost surely under the intended sampling laws.
Example. For \(X_i\sim N(\theta,\sigma^2)\) with known \(\sigma\), the flat prior \(\Pi(\mathrm{d}\theta)=\mathrm{d}\theta\) gives \[ \theta\mid X^{(n)}\sim N(\bar X,\sigma^2/n), \] which is proper for \(n\ge1\).
Weakly informative priors
Weakly informative priors regularize extreme parameter values without attempting to encode detailed prior knowledge.
Remark. A prior may be weak on the natural scale but strong after transformation. In hierarchical and high-dimensional models, weakly informative priors are often used to prevent separation, nonidentifiability, or unrealistic tail behavior.
Prior predictive checking
A prior can be assessed before seeing the data by simulating from the prior predictive distribution.
Proposition. If the prior predictive puts substantial mass on scientifically impossible data, then either the likelihood, the prior, or the parameterization is misspecified for the intended analysis.
Proof. The prior predictive is the distribution of data under the assumed joint Bayesian model. Events with large prior predictive probability are therefore events the model declares plausible before observation.
Jeffreys prior from information volume
Theorem. For a regular \(d\)-dimensional model with Fisher information \(I_\theta(\theta)\), Jeffreys prior \[ \pi_J(\theta)\propto|I_\theta(\theta)|^{1/2} \] is invariant under smooth one-to-one reparameterization \(\phi=g(\theta)\).
Proof. Let \(J=\partial\theta/\partial\phi^\top\). Scores transform as \(\dot\ell_\phi=J^\top\dot\ell_\theta\), so \[ I_\phi(\phi)=J^\top I_\theta(\theta)J. \] Therefore \[ |I_\phi(\phi)|^{1/2} =|\det J|\,|I_\theta(\theta)|^{1/2}, \] which is exactly the density change-of-variables rule.
Example.
- For \(X\sim\operatorname{Bernoulli}(p)\), \[ I(p)=\frac1{p(1-p)},\qquad \pi_J(p)\propto p^{-1/2}(1-p)^{-1/2}, \] the proper \(\operatorname{Beta}(1/2,1/2)\) prior.
- For \(X\sim N(\mu,\sigma^2)\) with \(\mu\) known, \[ I(\sigma)=\frac2{\sigma^2}, \qquad \pi_J(\sigma)\propto\frac1\sigma. \]
- For both \(\mu\) and \(\sigma\) unknown, the joint Jeffreys-rule prior is proportional to \(\sigma^{-2}\), while the independence-Jeffreys prior is proportional to \(\sigma^{-1}\). “Noninformative” is therefore not unique in multiparameter models.
Posterior propriety with improper priors
Theorem. For \[ X_i\mid\mu,\sigma^2\stackrel{\mathrm{iid}}{\sim}N(\mu,\sigma^2), \qquad \pi(\mu,\sigma)\propto\frac1\sigma, \] the posterior is proper if \(n\ge2\) and the observations are not all equal.
Proof. Integrating over \(\mu\), \[ \int_{\mathbb R} \exp\left\{-\frac{\sum_i(X_i-\mu)^2}{2\sigma^2}\right\} \mathrm{d}\mu =\sqrt{\frac{2\pi\sigma^2}{n}} \exp\left\{-\frac{S}{2\sigma^2}\right\}, \] where \(S=\sum_i(X_i-\bar X)^2>0\). Including the likelihood factor \(\sigma^{-n}\) and prior \(\sigma^{-1}\) leaves an integral proportional to \[ \int_0^\infty \sigma^{-n}e^{-S/(2\sigma^2)}\,\mathrm{d}\sigma. \] It is finite near zero because \(S>0\), and finite at infinity exactly when \(n>1\).
Improper-prior limitation. An improper prior has no prior predictive distribution and its arbitrary multiplicative constant does not cancel across models. Ordinary Bayes factors are therefore undefined unless compatible constants are supplied by a proper-prior construction, intrinsic Bayes factor, or fractional method.
Weakly informative scale priors
A weakly informative prior is calibrated on an interpretable data or effect scale. For a logistic coefficient, for example, a centered normal or Student prior should be judged through plausible odds ratios \[ e^\beta \] after covariates have meaningful scaling. For a standard deviation \(\tau\), half-normal and half-Student priors make behavior near zero and in the upper tail explicit.
Proposition. If \(\tau\sim\operatorname{half\text{-}Cauchy}(s)\), then \[ P(\tau>M)=1-\frac2\pi\arctan(M/s) \sim\frac{2s}{\pi M}. \] Thus the prior permits very large scales with polynomial tail probability and may yield unstable posterior moments when likelihood information is weak.
Proof. The half-Cauchy distribution function is \[ F(M)=\frac2\pi\arctan(M/s). \] Use \(\arctan x=\pi/2-x^{-1}+o(x^{-1})\).
Local and global sensitivity
Proposition (prior perturbation identity). Let \[ \pi_\varepsilon(\theta) \propto e^{\varepsilon h(\theta)}\pi_0(\theta). \] For an integrable posterior functional \(g\), \[ \left. \frac{\mathrm{d}}{\mathrm{d}\varepsilon} E_{\pi_\varepsilon}\{g(\theta)\mid x\} \right|_{\varepsilon=0} =\operatorname{Cov}_{\pi_0} \{g(\theta),h(\theta)\mid x\}. \]
Proof. Differentiate the ratio \[ \frac{E_{\pi_0}\{g(\theta)e^{\varepsilon h(\theta)}L(\theta;x)\}} {E_{\pi_0}\{e^{\varepsilon h(\theta)}L(\theta;x)\}} \] at zero. The quotient rule gives posterior covariance.
This identity identifies directions of prior sensitivity, while refitting a scientifically plausible family of priors reveals global changes. Sensitivity analysis is most useful for decisions, tail probabilities, and weakly identified components, not merely posterior means that are already data-dominated.