Classical to sparse
This page is a map for answering literature questions in seminars. It does not replace reading the papers; it records what role each line of work plays in a defense of sparse Bayesian asymptotics.
Proposition. Classical parametric BvM proves posterior normality in fixed dimension. Semiparametric BvM studies finite-dimensional functionals in infinite-dimensional models. Sparse BvM studies a finite-dimensional parameter whose effective dimension is unknown and selected from a high-dimensional ambient space.
Remark. Castillo-Nickl type results are central for understanding when Bayesian credible sets have frequentist meaning in nonparametric and high-dimensional settings. Ghosal-Ghosh-van der Vaart type contraction theory supplies the testing-prior-mass machinery behind concentration. Spokoiny-style finite-sample likelihood theory supplies uniform quadratic expansions beyond classical fixed-dimensional asymptotics.
Comparison questions
Definition. A comparison with an existing BvM theorem should identify four objects: the model class, the prior class, the metric or functional receiving Gaussian approximation, and the regime of \(n,p,s\).
Proposition. For a sparse GLM BvM result, the natural comparison axes are: Gaussian linear model versus GLM likelihood, exact support recovery versus marginal functional normality, fixed support size versus growing sparsity, and asymptotic LAN versus nonasymptotic likelihood control.
Proof. Each axis changes one mathematical obstruction. GLM likelihoods require curvature control depending on the link. Support uncertainty changes mixture weights. Growing sparsity changes normal approximation dimension. Nonasymptotic control changes the statement from pointwise LAN to uniform high-probability quadratic approximation.
Questions that organize the literature
Definition. Sparse BvM papers differ by target: full vector posterior, low-dimensional linear functional, prediction, selected model posterior, or debiased one-step marginal.
Proposition. The strongest oracle BvM results need exact or near-exact model recovery; functional BvM results can hold under weaker conditions if the functional is sufficiently regular and debiased.
Remark. A useful reading order is: posterior contraction, selection consistency, LAN on sparse neighborhoods, BvM on the oracle model, then variants that relax selection by targeting functionals.
Four mathematical objects to separate
Definition. Sparse posterior theory can target:
- the parameter law \(\Pi(\beta\mid X)\);
- the support law \(\Pi(S\mid X)\);
- a functional pushforward \(\Pi(a^\top\beta\in\cdot\mid X)\);
- the predictive law \[ p(\widetilde y\mid X) =\int p_\beta(\widetilde y)\Pi(\mathrm{d}\beta\mid X). \]
Each lives in a different metric and can have a different asymptotic limit.
Canonical implication chain
Proposition. A typical oracle BvM proof decomposes into:
\[ \text{testing + prior mass} \Longrightarrow \text{contraction}, \] \[ \text{contraction + complexity prior} \Longrightarrow \text{dimension control}, \] \[ \text{dimension control + beta-min} \Longrightarrow \text{selection}, \] \[ \text{selection + uniform sparse LAN + slab flatness} \Longrightarrow \text{oracle BvM}. \] None of the reverse implications holds in general.Explanation. Each arrow adds a new property: local metric concentration, combinatorial control, testing separation of zero and nonzero coordinates, and second-order likelihood shape. Counterexamples are obtained by respectively using diffuse model-size priors, sub-threshold signals, or nonregular likelihoods while retaining earlier properties.
Regime table
For every theorem, record the asymptotic regime:
- sample size \(n\), ambient dimension \(p_n\), and sparsity \(s_n\);
- fixed or random design and its normalization;
- known or unknown noise/dispersion;
- minimum signal and restricted eigenvalue constants;
- slab scale and model-size prior;
- pointwise or uniform truth class;
- total variation, Wasserstein, bounded-Lipschitz, or moment convergence.
A statement such as “the posterior is asymptotically normal” is incomplete until all seven coordinates are specified.
Three common BvM forms
Definition.
- Oracle conditional BvM: \[ \Pi\{\sqrt n(\beta_{S_0}-\widehat\beta_{S_0}) \in\cdot\mid S_0,X\} \approx N(0,I_{S_0}^{-1}). \]
- Mixture BvM: \[ \Pi(\mathrm{d}\beta\mid X) \approx\sum_Sw_SN_S(\widehat\beta_S,I_S^{-1}/n). \]
- Functional BvM: \[ \Pi\{\sqrt n(\psi(\beta)-\widehat\psi) \in\cdot\mid X\} \approx N(0,V_\psi). \]
The first becomes a full posterior result only after selection consistency. The second preserves model uncertainty. The third can hold when the first two fail, provided nuisance bias is removed.
Proof obligations by result type
Proposition.
- A contraction paper must provide prior thickness, tests, and sieve complexity.
- A selection paper must compare posterior odds over underfitted and overfitted models separately.
- An oracle BvM paper must prove uniform local quadratic expansion and prior flatness on the selected model.
- A functional BvM paper must identify an efficient or debiasing direction and control product remainders.
- A variational BvM paper must additionally control approximation and optimization gaps at the BvM scale.
Underfitting and overfitting arguments
Lemma. In sparse model selection, wrong supports split into:
- underfitted supports \(S\nsupseteq S_0\), defeated by likelihood separation proportional to omitted signal;
- overfitted supports \(S\supsetneq S_0\), defeated by model prior and marginal-likelihood dimension penalties.
This split is mathematically useful because beta-min controls the first class, while complexity priors and slab concentration control the second. A proof that bounds only all wrong models together often hides which assumption pays which combinatorial cost.
Uniformity near the selection boundary
Remark. Pointwise oracle BvM can hold for every fixed sparse vector with nonzero coefficients while failing uniformly as a coefficient approaches zero. At the \(n^{-1/2}\) or \(\sqrt{\log p/n}\) boundary, the limit is a nonregular mixture between inclusion and exclusion. Honest inference must either retain model uncertainty, weaken uniformity, or impose beta-min separation.
Reading record for one paper
A useful technical record contains:
- model and true parameter class;
- prior, including all \(n,p\)-dependent scaling;
- posterior target and convergence metric;
- contraction and model-size lemmas;
- LAN radius and remainder bound;
- selection or debiasing mechanism;
- limiting center and covariance;
- coverage consequence and its uniformity;
- one regime where an assumption fails.
This converts a literature list into a comparable theorem map and makes gaps between papers visible.