Hanqing Li Hanqing Li 李汉青

Oracle posterior normality

Sparse BvM theory tries to recover the fixed-dimensional BvM theorem after the posterior has learned a low-dimensional active set. The target is usually an oracle Gaussian law on the true or selected coordinates.

Definition. Let \(S_0\) be the true support with \(|S_0|=s_0\). An oracle sparse BvM statement has the form \[ \left\|\Pi\left(\sqrt n(\beta_{S_0}-\hat\beta_{S_0})\in\cdot,\ \beta_{S_0^c}=0\mid X^{(n)}\right)-N(0,I_{S_0}^{-1})\otimes\delta_0\right\|_{\mathrm{TV}}\to0. \]

Theorem. A typical sparse BvM proof requires: posterior concentration on models of size \(O(s_0)\), negligible posterior mass on wrong supports or negligible effect of wrong supports on the target functional, a uniform LAN expansion over sparse neighborhoods, and prior flatness on local \(n^{-1/2}\)-balls inside the active coordinates.

Proof. Decompose the posterior by support. On the oracle support, change variables \(h=\sqrt n(\beta_{S_0}-\beta_{0,S_0})\) and use LAN plus prior flatness. On nonoracle supports, use selection consistency or show their total contribution to the marginal law is negligible. Uniform concentration restricts the argument to sparse local sets where the quadratic approximation is valid.

Dimension growth

Remark. The phrase sparse BvM hides the most delicate point: \(s_0\) may grow. Gaussian approximation, determinant terms, and local prior flatness must remain uniform over dimensions increasing with \(n\).

Uniform LAN on sparse sets

Definition. Sparse LAN means that uniformly over supports \(|S|\le Cs_0\) and bounded \(h\in\mathbb R^S\), \[ \ell_n(\beta_0+h/\sqrt n)-\ell_n(\beta_0) =h^\top\Delta_{n,S}-{1\over2}h^\top I_S h+r_n(S,h), \] with \(\sup |r_n(S,h)|\to0\) in probability on the posterior local set.

Proposition. The main rate restrictions in sparse BvM usually come from controlling the LAN remainder, model-counting terms, and posterior leakage to large or wrong supports.

Proof. The Gaussian change of variables is local and supportwise. Summing over many supports introduces combinatorial factors, while Taylor remainders scale with sparse radius and dimension. The stated restrictions are exactly the places where dimension growth enters the fixed-dimensional BvM proof.

Oracle posterior normality

Theorem (oracle sparse BvM). Suppose:

  1. the posterior selects the true support: \[ \Pi(S=S_0\mid X^{(n)})\to1; \]
  2. \(|S_0|=s_0\) and the likelihood restricted to \(S_0\) is uniformly LAN;
  3. the slab density is locally flat and positive around \(\beta_{0,S_0}\);
  4. the restricted information \(I_{S_0}\) is nonsingular.

Then \[ \left\| \Pi\left( \sqrt n(\beta_{S_0}-\widehat\beta_{S_0}) \in\cdot\mid X^{(n)} \right) -N_{s_0}(0,I_{S_0}^{-1}) \right\|_{\mathrm{TV}} \to0, \] and posterior mass on \(\beta_{S_0^c}=0\) tends to one.

Proof. Conditional on \(S=S_0\), this is a growing-dimensional parametric BvM theorem based on uniform LAN and prior flatness. Selection consistency makes the total posterior differ from its oracle-support conditional posterior by at most \[ 2\Pi(S\ne S_0\mid X) \] in total variation. This term vanishes.

Uniform sparse LAN remainder

Proposition. Suppose third derivatives of the log likelihood satisfy, on every \(s\)-sparse direction, \[ \left| \nabla^3\ell_n(\beta)[u,u,u] \right| \le Cn\|u\|_2^3. \] For local radius \[ \|h\|_2\lesssim\sqrt{s\log p}, \qquad \beta=\beta_0+h/\sqrt n, \] the Taylor remainder is bounded by \[ C\frac{(s\log p)^{3/2}}{\sqrt n}. \] Hence a sufficient uniform LAN condition is \[ \frac{(s\log p)^3}{n}\to0. \]

Proof. Third-order Taylor remainder is at most \[ Cn\left\|\frac h{\sqrt n}\right\|_2^3 =C\frac{\|h\|_2^3}{\sqrt n}. \] Substitute the sparse posterior radius.

Sharper self-concordance, design structure, or norm-specific arguments can weaken this condition. It should be derived from the actual likelihood rather than quoted as universal.

Mixture over plausible supports

Theorem (mixture BvM form). If exact support recovery fails but posterior mass concentrates on a collection \(\mathcal S_n\) of small supports and a uniform BvM theorem holds on each, then the posterior can be approximated by \[ \sum_{S\in\mathcal S_n} \widehat w_S N_S\left( \widehat\beta_S,\, \frac1nI_S^{-1} \right), \] with point masses at zero outside \(S\), rather than by one full Gaussian.

This mixture can remain non-Gaussian for coordinates whose inclusion probabilities do not converge to zero or one. Marginal credible intervals may contain an atom at zero plus a continuous component; replacing them by one normal interval discards model uncertainty.

Low-dimensional functional BvM without selection

Theorem (functional template). Let \[ \psi(\beta)=a^\top\beta \] for a fixed loading \(a\). Suppose there is a sparse approximate inverse-information direction \(u_0\) satisfying \[ I(\beta_0)u_0=a, \] posterior contraction gives \[ \|\beta-\beta_0\|_1 =O_P\left( s_0\sqrt{\frac{\log p}{n}} \right), \] and orthogonalization makes the likelihood remainder \(o_P(n^{-1/2})\). Then a suitably debiased posterior marginal may satisfy \[ \sqrt n\{\psi(\beta)-\widehat\psi\} \mid X \rightsquigarrow N(0,a^\top I^{-1}a) \] even without exact support recovery.

Proof outline. Shift the parameter along the least-favorable direction \(u_0/\sqrt n\). The orthogonal score \[ u_0^\top\dot\ell_{\beta_0} \] has variance \(a^\top I^{-1}a\). Sparse contraction and product-rate conditions control nuisance coordinates and LAN remainder. Prior stability under the shift gives the posterior Laplace-transform limit.

Prior flatness on sparse neighborhoods

Definition. A local slab-flatness requirement is \[ \sup_{\substack{S\in\mathcal S_n\\ \|h\|\le M_n}} \left| \log \frac{g_S(\beta_{0,S}+h/\sqrt n)} {g_S(\beta_{0,S})} \right| =o(1). \] It prevents the prior from contributing a first-order score on posterior local scales.

Strong continuous shrinkage can violate flatness near nonzero coefficients, while diffuse slabs can hurt model selection through a large Occam penalty. Sparse BvM therefore demands a careful balance between enough local flatness for Gaussian shape and enough concentration for support learning.

Dimension, sparsity, and target hierarchy

Claims increase in strength:

  1. prediction or \(\ell^2\) contraction;
  2. posterior dimension control;
  3. support recovery;
  4. Gaussianity conditional on one support;
  5. oracle full-vector BvM;
  6. uniform BvM over a sparse class.

A theorem at one level does not imply the next. In particular, minimax contraction at \(\sqrt{s\log p/n}\) permits uncertainty among many supports and does not justify oracle covariance.