Sparse Bayesian inference
For seminar preparation, author names should be attached to mathematical roles rather than memorized as a flat list.
Remark. Castillo and Nickl are central for BvM and uncertainty in high-dimensional and nonparametric Bayes. Ghosal, Ghosh, and van der Vaart anchor posterior contraction theory. Spokoiny is a key reference for finite-sample likelihood expansions. Rousseau, Alquier, Petrone, Grünwald, and Chopin connect Bayesian asymptotics, PAC-Bayes, misspecification, computation, and model averaging themes.
Variational inference and high-dimensional statistics
Remark. Blei, Jordan, and Broderick are central names for variational inference and scalable Bayes. van de Geer and Wainwright are central for high-dimensional statistics, oracle inequalities, restricted eigenvalues, and sparse estimation. Hastie connects statistical learning, regularization, and applied sparse modeling.
Definition. A useful literature map records: problem, model, prior or estimator, main theorem, assumptions, and limitation. This format turns references into answers to questions such as what is new, what fails, and what comparison is fair.
How to classify a paper quickly
Definition. A useful map records four coordinates: model class, prior or approximation family, target of inference, and metric of validity.
Proposition. Most modern Bayesian theory papers can be read as proving one of three implications: prior thickness plus tests imply contraction; contraction plus local quadratic expansion implies Gaussian approximation; or variational/projection error plus posterior control implies approximate decision validity.
Remark. For personal research notes, record which implication a paper proves before recording the technical lemmas. That prevents a literature map from becoming an unstructured bibliography.
Coordinates of a theorem
Definition. A complete theorem record has the tuple \[ \mathfrak T= (\mathcal E_n,\Theta_n,\theta_{0,n}, \Pi_n,\mathcal Q_n,\psi_n,d_n,r_n, \mathcal A_n,\mathcal C_n), \] where:
- \(\mathcal E_n\) is the statistical experiment;
- \(\Theta_n\) and \(\theta_{0,n}\) are parameter space and truth;
- \(\Pi_n\) is the prior and \(\mathcal Q_n\) any approximation family;
- \(\psi_n\) is the inferential target;
- \(d_n\) and \(r_n\) are convergence metric and rate;
- \(\mathcal A_n\) are assumptions;
- \(\mathcal C_n\) is the conclusion and its uniformity class.
Two papers are directly comparable only after these coordinates are aligned. A faster rate in a weaker norm or for a smaller truth class is not automatically a stronger theorem.
Five recurring proof engines
Proposition. Most results in this note are assembled from five engines:
- disintegration and convexity: posterior formulas and Bayes actions;
- testing, prior mass, and entropy: posterior contraction;
- LAN and change of variables: BvM and Laplace limits;
- restricted geometry and model counting: sparse contraction and selection;
- variational comparison and transport: approximate posterior risk.
A paper's novelty often lies not in inventing a sixth engine, but in making one engine work for a new likelihood, prior, target, or growing-dimensional regime.
Claim hierarchy
Definition. Claims should be ordered by logical strength:
\[ \text{posterior propriety} \not\Rightarrow \text{consistency} \not\Rightarrow \text{rate optimality} \not\Rightarrow \text{BvM} \not\Rightarrow \text{uniform coverage}. \] For sparse problems, \[ \text{prediction contraction} \not\Rightarrow \text{support recovery} \not\Rightarrow \text{oracle BvM}. \] For approximations, \[ \text{algorithmic convergence} \not\Rightarrow \text{small posterior divergence} \not\Rightarrow \text{decision calibration}. \]Every arrow can fail because the right-hand claim introduces an additional requirement: identification, rate balance, second-order shape, uniform bias control, beta-min, or approximation calibration.
Assumption ledger
Classify assumptions by function:
- identification: distinct parameters yield distinguishable laws;
- local geometry: information, restricted curvature, differentiability;
- global complexity: entropy, sieve tails, model-size priors;
- prior thickness: local mass and support;
- bias control: smoothness, orthogonality, undersmoothing;
- algorithmic access: mixing, optimization, transport expressivity;
- uniformity: beta-min, self-similarity, positivity, bounded design.
This ledger prevents a technical condition from being copied into a new project without understanding which failure it blocks.
Counterexample field
Proposition. Every theorem record should include one nearest failure regime. Examples are:
- truth outside prior support for consistency;
- insufficient KL mass for contraction;
- boundary or singular information for BvM;
- sub-threshold signals for support recovery;
- nuisance bias larger than \(n^{-1/2}\) for functional BvM;
- omitted modes for reverse-KL variational inference;
- infinite importance-weight variance for correction.
A theorem is better understood by the nearest counterexample than by a long unclassified assumption list.
Research map for sparse BvM
For a high-dimensional posterior-normality project, the dependency chain is:
- statistical learning for concentration, compatibility, oracle inequalities, and debiasing;
- sparse priors and shrinkage for prior dimension and signal geometry;
- posterior contraction for localization;
- LAN for quadratic likelihood shape;
- sparse BvM for supportwise or functional Gaussian limits.
Research map for posterior compression and model averaging
A compression or predictor-redundancy project should separate:
- posterior target and exact reference law;
- compression map, model prior, or variational family;
- local likelihood or density-ratio error;
- predictive and decision loss;
- algorithmic optimization error;
- asymptotic preservation, such as contraction or BvM.
The relevant pages are Bayesian model averaging, dilution and projection predictive inference, and posterior compression and density-ratio diagnostics.
One-page paper record
For each paper, record:
- one-sentence problem and why existing theory fails;
- precise model, prior, and asymptotic regime;
- main theorem in mathematical notation;
- three indispensable assumptions and their roles;
- proof decomposition into reusable lemmas;
- comparison theorem under aligned assumptions;
- counterexample or excluded regime;
- connection to a current research question.
This is detailed enough to support an oral explanation, reproduce a proof, and identify a genuine extension without turning the note into an unstructured bibliography.