Exponential-family form
Bayesian GLMs combine prior regularization with likelihoods whose curvature depends on the link and variance function. In high dimension, the local likelihood geometry must be controlled on sparse cones.
Definition. A one-parameter exponential-family GLM has density \[ p_\beta(y_i\mid x_i)=\exp\left\{ {y_i\theta_i-b(\theta_i)\over a(\phi)}+c(y_i,\phi)\right\}, \] with mean \(\mu_i=b'(\theta_i)\), variance \(a(\phi)b''(\theta_i)\), and link \(g(\mu_i)=x_i^\top\beta\). The canonical link is \(g=(b')^{-1}\), so \(\theta_i=x_i^\top\beta\).
Proposition. For canonical links, the score and Hessian are \[ \dot\ell_n(\beta)=\sum_i x_i\{y_i-b'(x_i^\top\beta)\}/a(\phi),\quad -\ddot\ell_n(\beta)=\sum_i x_ix_i^\top b''(x_i^\top\beta)/a(\phi). \]
Proof. Differentiate the log likelihood with respect to \(x_i^\top\beta\), then multiply by \(x_i\). Differentiating once more gives the weighted Gram matrix with weights \(b''(x_i^\top\beta)/a(\phi)\).
High-dimensional curvature
Definition. A sparse GLM restricted curvature condition requires \[ \Delta^\top\{-n^{-1}\ddot\ell_n(\beta)\}\Delta\ge\kappa\|\Delta_S\|_2^2 \] for \(\Delta\) in a cone \(\|\Delta_{S^c}\|_1\le c\|\Delta_S\|_1\) and for \(\beta\) in a sparse local neighborhood.
Remark. Logistic regression can lose curvature under separation; Poisson regression can have exploding curvature if \(x_i^\top\beta\) is uncontrolled. Sparse BvM arguments must therefore combine design conditions with parameter localization.
Logistic and Poisson local curvature
Proposition. In logistic regression the weight is \(w_i(\beta)=p_i(\beta)\{1-p_i(\beta)\}\le1/4\). In Poisson regression the weight is \(w_i(\beta)=\exp(x_i^\top\beta)\). Hence logistic curvature can vanish under separation, while Poisson curvature can explode without localization.
Proof. The Hessian of a canonical GLM is \(X^\top W X\), with \(W_{ii}=b''(x_i^\top\beta)\). Substitute \(b(\eta)=\log(1+e^\eta)\) and \(b(\eta)=e^\eta\) for logistic and Poisson regression.
Posterior concentration before normality
Remark. A BvM theorem is a second-order statement. It normally requires a first-order contraction theorem first, because the posterior must lie in the region where sparse curvature and Taylor expansion are valid.
Canonical GLM likelihood geometry
Definition. For fixed design rows \(x_i^\top\), \[ \ell_n(\beta) =\sum_{i=1}^n \{y_ix_i^\top\beta-b(x_i^\top\beta)\} \] up to dispersion and parameter-free terms. The score and negative Hessian are \[ U_n(\beta) =X^\top\{y-\mu(\beta)\}, \] \[ H_n(\beta) =X^\top W(\beta)X, \qquad W_{ii}(\beta)=b''(x_i^\top\beta). \]
Proposition (local quadratic bounds). If on a neighborhood \(\mathcal B\), \[ 0<m\le b''(x_i^\top\beta)\le M<\infty \] for all \(i,\beta\in\mathcal B\), then \[ \frac m2\|X(\beta-\beta_0)\|_2^2 \le -\ell_n(\beta)+\ell_n(\beta_0) +U_n(\beta_0)^\top(\beta-\beta_0) \le \frac M2\|X(\beta-\beta_0)\|_2^2. \]
Proof. Apply Taylor's theorem along the segment between \(\beta_0\) and \(\beta\): \[ -\ell_n(\beta)+\ell_n(\beta_0) +U_n(\beta_0)^\top(\beta-\beta_0) =\frac12(\beta-\beta_0)^\top X^\top W(\widetilde\beta)X (\beta-\beta_0). \] Bound the diagonal weights.
Logistic curvature loss
Proposition. For logistic regression, \[ b''(\eta) =\frac{e^\eta}{(1+e^\eta)^2} \le\frac14, \] and \(b''(\eta)\to0\) as \(|\eta|\to\infty\). Thus global strong convexity is impossible without restricting linear predictors.
Proof. Differentiate \(b(\eta)=\log(1+e^\eta)\). The Bernoulli variance \(p(1-p)\) is at most \(1/4\) and tends to zero as \(p\to0\) or \(1\).
Posterior contraction must first confine \(\beta\) to a region where enough fitted probabilities remain away from zero and one on informative design directions. Sparse restricted curvature is therefore both local and truth-dependent.
Poisson curvature and tails
Proposition. For Poisson regression, \[ b''(\eta)=e^\eta. \] Curvature can vanish for very negative predictors and explode for very positive predictors. Uniform Taylor control requires bounds on \[ \max_i|x_i^\top(\beta-\beta_0)| \] and on the true intensity range.
Sparse GLM contraction
Theorem (schematic). Suppose \(\beta_0\) is \(s_0\)-sparse, the design satisfies a weighted restricted eigenvalue condition at truth, covariates are controlled, and a spike-and-slab prior has compatible complexity and slab mass. If \[ s_0\log p=o(n), \] then \[ \Pi\left( \frac{\|X(\beta-\beta_0)\|_2}{\sqrt n} >M\sqrt{\frac{s_0\log p}{n}} \middle|Y \right)\to0. \] Weighted restricted eigenvalues convert this to \(\ell^2\)-coefficient contraction.
Proof outline. Score concentration controls \[ \|X^\top(Y-\mu_0)\|_\infty \] at order \(\sqrt{n\log p}\). Local likelihood curvature converts prediction displacement into KL separation. Sparse covering entropy and model prior cost are \(s_0\log p\), while the slab supplies local mass. Apply the general contraction theorem within a predictor-bounded sieve.
Self-concordant likelihood control
Proposition. For logistic loss along direction \(u\), the third derivative satisfies \[ |\nabla^3\ell_n(\beta)[u,u,u]| \le \max_i|x_i^\top u|\, u^\top H_n(\beta)u. \]
Proof. The third derivative is \[ -\sum_i b'''(x_i^\top\beta)(x_i^\top u)^3. \] For logistic \(b\), \[ |b'''(\eta)|\le b''(\eta). \] Factor one \(\max_i|x_i^\top u|\) and recognize the Hessian quadratic form.
This generalized self-concordance controls Hessian change by local predictor displacement and can yield sharper LAN remainders than a global third-derivative bound.
High-dimensional BvM scale
On a support of size \(s\), posterior local radius is typically \[ \|\beta_S-\beta_{0,S}\|_2 =O_P(\sqrt{s/n}), \] conditional on support, or \[ O_P(\sqrt{s\log p/n}) \] uniformly across supports. A cubic remainder then suggests restrictions between \(s\), \(\log p\), maximum row norm, and \(n\). The exact BvM condition differs between logistic, Poisson, and Gaussian models because their curvature derivatives differ.
Separation and sparse priors
A proper sparse prior makes a logistic posterior proper under separation, but selection probabilities and moments can still be sensitive to slab tails. A thin slab may force finite estimates yet over-shrink true large coefficients; a very diffuse slab may create a large model-evidence penalty. Separation is therefore a regime in which prior choice remains first order rather than disappearing under BvM.
Unknown dispersion and nuisance parameters
For noncanonical or overdispersed GLMs, dispersion and nuisance functions alter score covariance and information. A sparse BvM theorem for \(\beta\) must either estimate them fast enough, prove orthogonality, or include them in the local Gaussian block. Plugging in a dispersion estimate without accounting for its uncertainty is a separate approximation.