Hanqing Li Hanqing Li 李汉青


Transport inequalities convert information divergence into geometric distance. Through Kantorovich duality they become exponential bounds for Lipschitz observables, and through gradient flows they connect entropy, Fisher information, curvature, and concentration.

Relative entropy and transport inequalities

Definition. For \(\nu,\mu\in\mathcal P(X)\), \[ \operatorname{KL}(\nu\,\|\,\mu) = \begin{cases} \displaystyle\int\log\!\left(\frac{\mathrm{d}\nu}{\mathrm{d}\mu}\right)\mathrm{d}\nu, &\nu\ll\mu,\\ +\infty,&\text{otherwise}. \end{cases} \] A measure \(\mu\) satisfies \(T_1(C)\) if \[ W_1(\nu,\mu)\le\sqrt{2C\,\operatorname{KL}(\nu\,\|\,\mu)} \] for every \(\nu\), and satisfies \(T_2(C)\) if \[ W_2^2(\nu,\mu)\le2C\,\operatorname{KL}(\nu\,\|\,\mu). \]

Since \(W_1\le W_2\), \(T_2(C)\) implies \(T_1(C)\). Pinsker's inequality, \[ \|\nu-\mu\|_{\mathrm{TV}}^2\le\frac12\operatorname{KL}(\nu\,\|\,\mu), \] is an analogous comparison in total variation, but it does not encode the geometry of the underlying space.

Entropy variational formula

Donsker--Varadhan formula. For \(\nu\ll\mu\), \[ \operatorname{KL}(\nu\,\|\,\mu) =\sup_f\left\{\int f\,\mathrm{d}\nu-\log\int e^f\,\mathrm{d}\mu\right\}, \] where the supremum may be taken over bounded measurable \(f\).

Proof. Let \(h=\mathrm{d}\nu/\mathrm{d}\mu\). For any \(f\), define the tilted law \(\mathrm{d}\mu_f=e^f\mathrm{d}\mu/\int e^f\mathrm{d}\mu\). Nonnegativity of \(\operatorname{KL}(\nu\,\|\,\mu_f)\) gives \[ \int f\,\mathrm{d}\nu-\log\int e^f\,\mathrm{d}\mu \le\int\log h\,\mathrm{d}\nu. \] Truncations of \(f=\log h\) approach equality.

\(T_1\) and sub-Gaussian observables

Bobkov--Götze characterization. A probability measure \(\mu\) satisfies \(T_1(C)\) if and only if every \(1\)-Lipschitz \(f\) satisfies \[ \log\int e^{\lambda(f-\mu f)}\,\mathrm{d}\mu \le\frac{C\lambda^2}{2} \qquad\text{for every }\lambda\in\mathbb R. \]

Proof from \(T_1\). For the exponentially tilted law \(\nu_\lambda\propto e^{\lambda f}\mu\), Kantorovich--Rubinstein duality gives \[ \nu_\lambda f-\mu f\le W_1(\nu_\lambda,\mu) \le\sqrt{2C\,\operatorname{KL}(\nu_\lambda\,\|\,\mu)}. \] Writing \(\Lambda(\lambda)=\log\mu e^{\lambda(f-\mu f)}\), one has \(\Lambda'(\lambda)=\nu_\lambda(f-\mu f)\) and \(\operatorname{KL}(\nu_\lambda\,\|\,\mu)=\lambda\Lambda'(\lambda)-\Lambda(\lambda)\). The resulting differential inequality integrates to \(\Lambda(\lambda)\le C\lambda^2/2\).

Reverse implication. For any \(\nu\), the entropy variational formula applied to \(\lambda(f-\mu f)\) gives \[ \lambda(\nu f-\mu f) \le\operatorname{KL}(\nu\,\|\,\mu)+\frac{C\lambda^2}{2}. \] Optimize over \(\lambda>0\), then take the supremum over \(1\)-Lipschitz \(f\) and use Kantorovich--Rubinstein duality.

Concentration. Under \(T_1(C)\), every \(1\)-Lipschitz \(f\) satisfies \[ \mu(f-\mu f\ge r)\le e^{-r^2/(2C)}. \] This follows from the exponential moment bound and Chernoff optimization.

Talagrand's Gaussian \(T_2\) inequality

Gaussian transport inequality. For the standard Gaussian measure \(\gamma=N(0,I_d)\) and every \(\nu\in\mathcal P_2(\mathbb R^d)\), \[ W_2^2(\nu,\gamma)\le2\operatorname{KL}(\nu\,\|\,\gamma). \]

Proof. It suffices by approximation to assume that \(\nu=f\gamma\) is smooth and positive and that the Brenier map \(T=\nabla u\) pushing \(\gamma\) to \(\nu\) is smooth with \(DT>0\). The Jacobian identity gives \[ f(T(x))e^{-|T(x)|^2/2}\det DT(x)=e^{-|x|^2/2}, \] hence \[ \log f(T(x)) =\frac{|T(x)|^2-|x|^2}{2}-\log\det DT(x). \] Because \(\log a\le a-1\) eigenvalue by eigenvalue, \[ \log\det DT\le\operatorname{tr}(DT-I). \] Therefore \[ \begin{aligned} \operatorname{KL}(\nu\,\|\,\gamma) &=\int\log f(T(x))\,\mathrm{d}\gamma(x)\\ &\ge\frac12\int|T-x|^2\,\mathrm{d}\gamma +\int x\cdot(T-x)\,\mathrm{d}\gamma -\int\operatorname{tr}(DT-I)\,\mathrm{d}\gamma. \end{aligned} \] Gaussian integration by parts gives \[ \int x\cdot(T-x)\,\mathrm{d}\gamma =\int\operatorname{div}(T-x)\,\mathrm{d}\gamma =\int\operatorname{tr}(DT-I)\,\mathrm{d}\gamma. \] The last two terms cancel. Since \(T\) is optimal, \[ \operatorname{KL}(\nu\,\|\,\gamma) \ge\frac12\int|T-x|^2\,\mathrm{d}\gamma =\frac12W_2^2(\nu,\gamma). \]

Logarithmic Sobolev inequality and \(T_2\)

Otto--Villani implication. If \(\mu\) satisfies the logarithmic Sobolev inequality \[ \operatorname{KL}(\nu\,\|\,\mu) \le\frac C2 I(\nu\,\|\,\mu), \] where \[ I(\nu\,\|\,\mu)=\int\left|\nabla\log\frac{\mathrm{d}\nu}{\mathrm{d}\mu}\right|^2\,\mathrm{d}\nu, \] then \(\mu\) satisfies \(T_2(C)\).

Proof architecture. Follow the entropy gradient flow from \(\nu\) toward \(\mu\). Along the flow, \[ -\frac{\mathrm{d}}{\mathrm{d}t}\operatorname{KL}(\nu_t\,\|\,\mu)=I(\nu_t\,\|\,\mu), \qquad |\dot\nu_t|=\sqrt{I(\nu_t\,\|\,\mu)}. \] Hence \[ W_2(\nu,\mu)\le\int_0^\infty\sqrt{I(\nu_t\,\|\,\mu)}\,\mathrm{d}t. \] The logarithmic Sobolev inequality forces exponential entropy decay and converts this integral into \(\sqrt{2C\,\operatorname{KL}(\nu\,\|\,\mu)}\).

The HWI inequality

HWI inequality. If \(\mu=e^{-V}\mathrm{d}x/Z\) with \(D^2V\ge\kappa I\), then \[ \operatorname{KL}(\nu\,\|\,\mu) \le W_2(\nu,\mu)\sqrt{I(\nu\,\|\,\mu)} -\frac{\kappa}{2}W_2^2(\nu,\mu). \]

Proof. Relative entropy is displacement \(\kappa\)-convex. Apply the convexity inequality along the geodesic from \(\nu\) to \(\mu\), divide by \(t\), and let \(t\downarrow0\). The directional derivative at \(\nu\) is the \(L^2(\nu)\) pairing of \(\nabla\log(\mathrm{d}\nu/\mathrm{d}\mu)\) with the geodesic velocity. Cauchy--Schwarz bounds it by \(W_2(\nu,\mu)\sqrt I\), yielding the formula.

Tensorization

If each \(\mu_i\) satisfies \(T_2(C_i)\) on \(X_i\), then the product measure satisfies \(T_2(\max_iC_i)\) for the squared product metric. The proof disintegrates a competitor coordinate by coordinate, applies the one-coordinate inequalities conditionally, and uses the chain rule for relative entropy. This dimension-free tensorization is a central reason transport inequalities yield concentration for high-dimensional product measures.