The entropy power inequality is the continuous counterpart of a volume-growth statement. It is one of the deepest structural inequalities in classical information theory.
Entropy power
Definition. For \(X\in\mathbb R^d\) with differential entropy \(h(X)\), define entropy power \[ N(X)=\frac1{2\pi e}\exp\left(\frac{2}{d}h(X)\right). \] For a Gaussian vector with covariance \(\sigma^2 I_d\), \(N(X)=\sigma^2\).
Entropy power inequality
Theorem. If \(X\) and \(Y\) are independent \(\mathbb R^d\)-valued random vectors with densities, then \[ N(X+Y)\ge N(X)+N(Y). \] Equality holds in the scalar nondegenerate case when \(X\) and \(Y\) are Gaussian.
Brunn-Minkowski analogy
Remark. The inequality resembles \(|A+B|^{1/d}\ge |A|^{1/d}+|B|^{1/d}\) for sets after translating volume into entropy power. This is why EPI is often viewed as a probabilistic Brunn-Minkowski theorem.
Consequences
Corollary. EPI implies Gaussian extremality statements and gives a short route to capacity formulas for Gaussian noise channels. It also connects to Fisher information through de Bruijn's identity and Stam's inequality.
Entropy power
Definition. For an absolutely continuous \(X\in\mathbb R^d\), its entropy power is \[ N(X)=\frac1{2\pi e} \exp\left\{\frac{2}{d}h(X)\right\}. \] If \(X\sim N(0,\sigma^2I_d)\), then \(N(X)=\sigma^2\).
Entropy power inequality
Theorem. If \(X,Y\in\mathbb R^d\) are independent with densities and finite entropies under standard regularity, then \[ N(X+Y)\ge N(X)+N(Y). \] Equality holds for Gaussian \(X,Y\) with proportional covariance matrices.
Equivalent normalized form
Proposition. For every \(0\le\lambda\le1\), \[ h(\sqrt\lambda X+\sqrt{1-\lambda}Y) \ge\lambda h(X)+(1-\lambda)h(Y) \] after scaling \(X,Y\) so that their entropy powers satisfy the optimizing proportion. More generally this concavity form is equivalent to EPI through rescaling and optimization.
Proof outline. Apply EPI to \(aX+bY\): \[ N(aX+bY)\ge a^2N(X)+b^2N(Y). \] Choose scales so the two terms contribute proportions \(\lambda\) and \(1-\lambda\), take logarithms, and use \(h(aX)=h(X)+d\log|a|\). Conversely, optimize the concavity inequality over scaling.
Proof through heat flow
Proof outline. Add independent Gaussian noise and consider \[ X_t=X+\sqrt t\,Z,\qquad Y_t=Y+\sqrt t\,Z'. \] De Bruijn's identity gives \[ \frac{\mathrm{d}}{\mathrm{d}t}h(X_t)=\frac12J(X_t). \] The Fisher-information inequality \[ \frac1{J(X_t+Y_t)} \ge\frac1{J(X_t)}+\frac1{J(Y_t)} \] controls the derivative of the entropy-power deficit. At large \(t\), Gaussian noise dominates and the inequality becomes equality in the limit. Integrating backward proves nonnegativity of the deficit. Approximation removes smoothness assumptions.
Stam's Fisher-information inequality
Theorem. For independent scalar \(X,Y\), \[ \frac1{J(X+Y)} \ge\frac1{J(X)}+\frac1{J(Y)}. \]
Proof. Let scores be \(\rho_X=X\)-density log derivatives. The convolution score identity gives \[ \rho_{X+Y}(X+Y)= \mathbb E\{\rho_X(X)\mid X+Y\} =\mathbb E\{\rho_Y(Y)\mid X+Y\}. \] For constants \(a+b=1\), \[ \rho_{X+Y}(X+Y)= \mathbb E\{a\rho_X(X)+b\rho_Y(Y)\mid X+Y\}. \] Conditional Jensen, independence, and zero score means give \(J(X+Y)\le a^2J(X)+b^2J(Y)\). Minimize over \(a\), obtaining the reciprocal sum.
Gaussian maximum entropy from EPI
Theorem. If \(X_1,X_2,\ldots\) are iid copies of a scalar \(X\) with variance \(\sigma^2\), then \[ h\left(\frac{X_1+\cdots+X_n}{\sqrt n}\right) \] is nondecreasing along powers of two and bounded above by \(\tfrac12\log(2\pi e\sigma^2)\).
Proof. Apply EPI to two independent normalized sums of size \(n\): \[ N\left(\frac{S_n+S_n'}{\sqrt{2n}}\right) \ge\frac12N(S_n/\sqrt n)+ \frac12N(S_n'/\sqrt n), \] so entropy power does not decrease. Variance stays \(\sigma^2\), and Gaussian maximum entropy gives the upper bound. Under regularity, the entropic central limit theorem strengthens this to convergence.
Vector covariance consequence
Proposition. Since \(N(X)\le\{\det\operatorname{Cov}(X)\}^{1/d}\) by Gaussian extremality, EPI links entropy growth under convolution to determinant inequalities for covariance. Equality throughout requires Gaussian summands with proportional covariance.
Conditional EPI
A conditional EPI requires a conditional independence structure. If \(X\perp\!\!\!\perp Y\mid U\), one can derive conditional entropy-power inequalities with care over averaging and exponentiation. Simply conditioning an unconditional EPI is invalid when conditional dependence remains.
Why EPI is stronger than variance addition
Variance always adds for independent centered variables: \(\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)\). Entropy power is at most variance, with equality only for Gaussian laws. EPI says convolution increases the Gaussian-equivalent spread by at least the sum of the individual entropy powers, a statement sensitive to the whole density rather than only second moments.