Forward and reverse KL projections solve different variational problems. Their geometric behavior is best understood through dual affine structures.
Two projection problems
The order of KL arguments changes the projection.
Definition. For a set \(\mathcal M\) of distributions and a distribution \(P\), the forward KL projection is \[ \operatorname*{argmin}_{Q\in\mathcal M}D(P\|Q), \] while the reverse KL projection is \[ \operatorname*{argmin}_{Q\in\mathcal M}D(Q\|P). \]
Remark. The terminology is not universal. What matters is always the order of the two KL arguments.
First-order condition
Projection is characterized by orthogonality of the divergence gradient to feasible directions.
Proposition. If \(Q_{\hat\theta}\) minimizes \(D(P\|Q_\theta)\) over a smooth model, then \[ \partial_i D(P\|Q_\theta)|_{\hat\theta}=0. \] Equivalently, \(\int \partial_i\log q_{\hat\theta}\,\mathrm{d}P=\int \partial_i\log q_{\hat\theta}\,\mathrm{d}Q_{\hat\theta}=0\) when the model is normalized.
Proof. Differentiate \(D(P\|Q_\theta)=\int\log p\,\mathrm{d}P-\int\log q_\theta\,\mathrm{d}P\). At an interior minimum the derivative vanishes. The second equality uses the score mean-zero identity under \(Q_{\hat\theta}\).
Mode seeking and mass covering
The asymmetry has visible consequences in approximation.
Proposition. If \(\mathcal M\) is a restricted unimodal family approximating a multimodal \(P\), minimizing \(D(Q\|P)\) penalizes placing mass where \(P\) is small, while minimizing \(D(P\|Q)\) penalizes failing to cover mass of \(P\).
Proof. Reverse KL integrates \(\log(Q/P)\) under \(Q\), so regions where \(Q>0\) and \(P\) is tiny are expensive; it can avoid a mode by putting little \(Q\)-mass there. Forward KL integrates \(\log(P/Q)\) under \(P\), so regions with \(P>0\) and tiny \(Q\) are expensive; it must cover the support carrying \(P\)-mass.
Pythagorean identities
In flat geometries, KL projections satisfy exact additive identities.
Remark. Projection onto an exponential family and projection onto a mixture family use dual geodesics. Orthogonality under the Fisher metric gives generalized Pythagorean theorems for KL divergence.
Fixing the terminology
For a target law \(P\) and model \(\mathcal Q\), define \[ Q_{\mathrm I}\in\arg\min_{Q\in\mathcal Q}D(Q\|P), \qquad Q_{\mathrm M}\in\arg\min_{Q\in\mathcal Q}D(P\|Q). \] We call these the I-projection and M-projection, respectively. The terms “forward” and “reverse” KL are used inconsistently across statistics and machine learning, so formulas rather than names should carry the meaning.
Moment matching for an exponential family
Theorem. Let \(q_\theta(x)=\exp\{\theta^\top T(x)-\psi(\theta)\}h(x)\) be minimal and regular. If \(P|T|<\infty\), any interior minimizer of \(D(P\|Q_\theta)\) satisfies \[ E_{Q_\theta}T=E_PT. \] Strict convexity of \(\psi\) makes it unique when the target moment lies in the interior mean-parameter space.
Proof. Terms independent of \(\theta\) drop out: \[ D(P\|Q_\theta) =\text{constant}-\theta^\top E_PT+\psi(\theta). \] Its gradient is \(\nabla\psi(\theta)-E_PT\), and its Hessian is \(\operatorname{Cov}_{Q_\theta}(T)\succ0\) by minimality.
Support behavior and mode selection
If \(D(P\|Q)<\infty\), \(Q\) must be positive wherever \(P\) is positive; missing target mass is infinitely penalized. By contrast, \(D(Q\|P)\) only integrates where \(Q\) puts mass. In constrained unimodal families approximating a separated mixture, this can favor one high-density mode. The familiar “mass-covering” versus “mode-seeking” language is therefore a support-weighting tendency, not a theorem that applies to every model class.
Example (Gaussian projection). Among all nonsingular Gaussians \(Q=N(m,S)\), minimizing \(D(P\|Q)\) yields \[ m=E_PX,\qquad S=\operatorname{Cov}_P(X), \] provided the moments exist. Minimizing \(D(Q\|P)\) has no such universal moment formula and depends on the detailed shape of \(\log p\).
Pythagorean inequalities
Theorem. If \(\mathcal C\) is convex in mixture coordinates and \(Q^*=\arg\min_{Q\in\mathcal C}D(Q\|P)\), then under existence and differentiability, \[ D(Q\|P)\ge D(Q\|Q^*)+D(Q^*\|P), \qquad Q\in\mathcal C. \] Equality holds when \(\mathcal C\) is affine in the corresponding dual geometry.
This inequality is a Bregman projection result. Reversing KL also reverses which convexity structure supplies the clean identity. Nonconvex variational families may have multiple local projections, so neither uniqueness nor an exact Pythagorean theorem is automatic.