The Radon-Nikodym theorem is the measure-theoretic derivative theorem. It tells when one measure has a density with respect to another.
Absolute continuity of measures
One measure is absolutely continuous with respect to another if it ignores the same null sets.
Definition. For measures \(\nu,\mu\), write \(\nu\ll\mu\) if \(\mu(A)=0\Rightarrow \nu(A)=0\). A density of \(\nu\) with respect to \(\mu\) is a measurable \(f\ge0\) such that \(\nu(A)=\int_A f\,\mathrm{d}\mu\).
Proposition. If \(\nu(A)=\int_A f\,\mathrm{d}\mu\), then \(\nu\ll\mu\), and \(f\) is unique \(\mu\)-almost everywhere.
Proof. If \(\mu(A)=0\), then \(\int_A f\,\mathrm{d}\mu=0\). If \(f,g\) give the same measure, then \(\int_A(f-g)\,\mathrm{d}\mu=0\) for all \(A\). Taking \(A=\{f>g\}\) and \(A=\{g>f\}\) gives \(f=g\) almost everywhere.
Radon-Nikodym theorem
Absolute continuity is also sufficient for the existence of a density under sigma-finiteness.
Theorem. If \(\nu\) and \(\mu\) are sigma-finite measures and \(\nu\ll\mu\), then there exists a measurable \(f\ge0\) such that \(\nu(A)=\int_A f\,\mathrm{d}\mu\) for all \(A\). Write \(f=\mathrm{d}\nu/\mathrm{d}\mu\).
Proof. First suppose \(\mu,\nu\) are finite and put \(\rho=\mu+\nu\). On the Hilbert space \(L^2(\rho)\), define \[ Tg=\int g\,\mathrm{d}\nu. \] By Cauchy-Schwarz, \[ |Tg| \le \nu(X)^{1/2}\left(\int|g|^2\,\mathrm{d}\nu\right)^{1/2} \le\nu(X)^{1/2}\|g\|_{L^2(\rho)}. \] Riesz representation gives \(h\in L^2(\rho)\) such that \(Tg=\int gh\,\mathrm{d}\rho\). Taking \(g=\mathbf1_A\) shows \[ \nu(A)=\int_Ah\,\mathrm{d}\rho. \] Because \(0\le\nu\le\rho\), testing sets where \(h<0\) or \(h>1\) gives \(0\le h\le1\) almost everywhere. Also \[ \mu(A)=\rho(A)-\nu(A)=\int_A(1-h)\,\mathrm{d}\rho. \] On \(H=\{h=1\}\), \(\mu(H)=0\), hence \(\nu(H)=0\) by \(\nu\ll\mu\), and therefore \(\rho(H)=0\). Thus \[ f=\frac{h}{1-h} \] is finite \(\rho\)-almost everywhere and \[ \int_Af\,\mathrm{d}\mu =\int_A\frac{h}{1-h}(1-h)\,\mathrm{d}\rho =\int_Ah\,\mathrm{d}\rho =\nu(A). \]
For sigma-finite \(\mu,\nu\), choose a disjoint measurable partition \(X=\bigsqcup_nE_n\) with \(\mu(E_n),\nu(E_n)<\infty\). Apply the finite result to the restrictions on each \(E_n\), obtaining \(f_n\). Then \(f=\sum_n f_n\mathbf1_{E_n}\) is measurable and gives the required density on every measurable set by countable additivity.
Corollary (signed version). If a sigma-finite signed measure \(\lambda\) satisfies \(\lambda\ll\mu\), then there is an integrable real \(f\) such that \[ \lambda(A)=\int_Af\,\mathrm{d}\mu. \] For finite \(\lambda\), \(\int|f|\,\mathrm{d}\mu=|\lambda|(X)\).
Proof. The Jordan parts \(\lambda^+,\lambda^-\) are both absolutely continuous with respect to \(\mu\). Apply the positive theorem to get densities \(f^+,f^-\) and set \(f=f^+-f^-\). Mutual singularity of the Jordan parts gives \(|f|=f^++f^-\) almost everywhere, yielding the variation identity.
Chain rule and change of measure
Radon-Nikodym derivatives behave like ordinary derivatives.
Proposition. If \(\lambda\ll\nu\ll\mu\), then \[ \frac{\mathrm{d}\lambda}{\mathrm{d}\mu} =\frac{\mathrm{d}\lambda}{\mathrm{d}\nu}\frac{\mathrm{d}\nu}{\mathrm{d}\mu} \quad \mu\text{-a.e.} \]
Proof. For every measurable \(A\), \[ \lambda(A)=\int_A \frac{\mathrm{d}\lambda}{\mathrm{d}\nu}\,\mathrm{d}\nu =\int_A \frac{\mathrm{d}\lambda}{\mathrm{d}\nu}\frac{\mathrm{d}\nu}{\mathrm{d}\mu}\,\mathrm{d}\mu. \] Uniqueness of Radon-Nikodym derivatives gives the identity.
Proposition (reciprocal derivative). If \(\nu\ll\mu\) and \(\mu\ll\nu\), then \[ \frac{\mathrm{d}\mu}{\mathrm{d}\nu} =\left(\frac{\mathrm{d}\nu}{\mathrm{d}\mu}\right)^{-1} \quad \nu\text{-almost everywhere}, \] and \(\mathrm{d}\nu/\mathrm{d}\mu>0\) \(\mu\)-almost everywhere.
Proof. Apply the chain rule to \(\mu\ll\nu\ll\mu\): \[ 1=\frac{\mathrm{d}\mu}{\mathrm{d}\mu} =\frac{\mathrm{d}\mu}{\mathrm{d}\nu}\frac{\mathrm{d}\nu}{\mathrm{d}\mu}. \] The product identity also rules out a zero density on a set of positive \(\mu\)-measure.
Lebesgue differentiation viewpoint
Densities are local ratios of masses in Euclidean spaces.
Theorem. If \(\nu\ll m\) on \(\mathbb R^d\) with density \(f\), then for \(m\)-almost every \(x\), \[ f(x)=\lim_{r\downarrow0}\frac{\nu(B(x,r))}{m(B(x,r))}. \]
Proof. Since \(\nu(B(x,r))=\int_{B(x,r)}f\,\mathrm{d}m\), the ratio is the average of \(f\) over the ball. The Lebesgue differentiation theorem says these averages converge to \(f(x)\) almost everywhere.
Likelihood ratios as measure derivatives
A statistical density need not be a density with respect to Lebesgue measure; it is always relative to a dominating measure.
Remark. If \(P_\theta\ll P_{\theta_0}\), the likelihood ratio is \[ L_{\theta,\theta_0} =\frac{\mathrm{d}P_\theta}{\mathrm{d}P_{\theta_0}}, \qquad \mathbb E_{\theta_0}[L_{\theta,\theta_0}]=1. \] For any integrable statistic \(T\), \[ \mathbb E_\theta[T] =\mathbb E_{\theta_0}[T L_{\theta,\theta_0}]. \] This identity is the rigorous change-of-measure formula behind likelihood theory, importance sampling, and contiguity.