The Fisher information metric is the intrinsic Riemannian metric of a regular statistical model. It measures the squared length of infinitesimal distributional change through score covariance.
Metric tensor
The Fisher metric is the \(L^2(P_\theta)\) inner product of tangent scores.
Definition. The Fisher information metric is \[ g_{ij}(\theta)=\mathbb E_\theta[s_i s_j] =\int \partial_i\ell_\theta\,\partial_j\ell_\theta\,p_\theta\,\mathrm{d}\mu. \] For tangent vectors \(u,v\), \(g_\theta(u,v)=\mathbb E_\theta[(u^is_i)(v^js_j)]\).
Proposition. The metric is symmetric and positive semidefinite. It is positive definite iff no nonzero tangent direction has zero score in \(L^2(P_\theta)\).
Proof. Symmetry follows from multiplication. For any \(u\), \[ g_\theta(u,u)=\mathbb E_\theta(u^is_i)^2\ge0. \] It is zero exactly when \(u^is_i=0\) almost surely.
Expected Hessian identity
Under regularity, information is both score variance and expected log-likelihood curvature.
Proposition. If differentiation under the integral is valid, then \[ g_{ij}(\theta)=-\mathbb E_\theta[\partial_i\partial_j\ell_\theta]. \]
Proof. The score has mean zero. Differentiate \(\mathbb E_\theta s_i=0\) with respect to \(\theta^j\). The expectation derivative identity gives \[ 0=\mathbb E_\theta(\partial_j s_i)+\mathbb E_\theta(s_is_j). \] Since \(\partial_js_i=\partial_j\partial_i\ell_\theta\), rearranging gives the formula.
Tensor transformation law
The Fisher matrix changes by the usual pullback rule, so the metric is coordinate-free.
Theorem. Under a reparametrization \(\theta=\theta(\phi)\), \[ g^{(\phi)}_{ab} =\frac{\partial\theta^i}{\partial\phi^a} \frac{\partial\theta^j}{\partial\phi^b}g^{(\theta)}_{ij}. \]
Proof. The score transformation is \(s_a=(\partial\theta^i/\partial\phi^a)s_i\). Taking the \(L^2(P)\) inner product of \(s_a\) and \(s_b\) gives the displayed transformation.
Information and local identifiability
Degenerate Fisher information signals directions invisible to first order.
Remark. A singular Fisher metric may arise from overparametrization, mixture singularities, boundary points, or nonregular models. Classical information geometry usually starts away from these singularities.
Intrinsic definition and regularity
Definition. For tangent vectors \(u,v\in T_\theta\mathcal M\), represented by scores \(\ell_u,\ell_v\in L_2^0(P_\theta)\), the Fisher metric is \[ g_\theta(u,v)=P_\theta(\ell_u\ell_v). \] In coordinates \(u=u^i\partial_i\), \(v=v^j\partial_j\), \[ g_\theta(u,v)=u^iv^jg_{ij}(\theta), \qquad g_{ij}(\theta)=P_\theta(\partial_i\log p_\theta\, \partial_j\log p_\theta). \]
This is a Riemannian metric only when the score map is injective. If \(g_\theta(u,u)=0\) for nonzero \(u\), the parametrization is locally unidentifiable and \(g\) is merely positive semidefinite.
Proposition (expected Hessian identity). If differentiation may pass twice under the integral and the support does not depend on \(\theta\), then \[ g_{ij}(\theta) =-P_\theta\{\partial_{ij}^2\log p_\theta(X)\}. \]
Proof. Since \(\int p_\theta\,\mathrm{d}\mu=1\), \[ 0=\partial_i\int p_\theta\,\mathrm{d}\mu =P_\theta(\partial_i\log p_\theta). \] Differentiate this identity with respect to \(\theta^j\). The derivative of an expectation contributes both the derivative of the integrand and its product with the score: \[ 0=P_\theta(\partial_{ij}^2\log p_\theta) +P_\theta(\partial_i\log p_\theta\,\partial_j\log p_\theta). \]
Pullbacks, products, and information loss
Proposition. Fisher information has the following structural properties.
- For a smooth submodel \(\theta=\theta(\eta)\) with Jacobian \(J=\partial\theta/\partial\eta^\top\), \(I_\eta=J^\top I_\theta J\).
- For \(n\) independent observations, \(I_\theta^{(n)}=nI_\theta\).
- If \(X\) and \(Y\) are conditionally independent given \(\theta\), then \(I_{(X,Y)}(\theta)=I_X(\theta)+I_Y(\theta)\).
- For a statistic \(T=T(X)\), \(I_T(\theta)\preceq I_X(\theta)\).
Proof. The first property is the score chain rule. Independence makes the joint score a sum; centered cross terms vanish, proving the second and third claims. Finally, the score of the induced law of \(T\) is \[ \dot\ell_\theta^\top(T) =E_\theta\{\dot\ell_\theta^X(X)\mid T\}. \] Conditional expectation is an orthogonal projection in \(L_2(P_\theta)\), so for every \(a\), \[ a^\top I_Ta =\|E(a^\top\dot\ell^X\mid T)\|_2^2 \le \|a^\top\dot\ell^X\|_2^2 =a^\top I_Xa. \]
Bernoulli geometry
For \(X\sim\operatorname{Bernoulli}(p)\), \[ \dot\ell_p(X)=\frac{X-p}{p(1-p)}, \qquad g_p=\frac{1}{p(1-p)}\,\mathrm{d}p^2. \] The angular coordinate \(\phi=2\arcsin\sqrt p\) satisfies \(\mathrm{d}\phi^2=\mathrm{d}p^2/\{p(1-p)\}\). Thus the apparently singular metric becomes Euclidean on \(\phi\in(0,\pi)\), and the Fisher--Rao distance is \[ d_{\mathrm{FR}}(p,q) =2\left|\arcsin\sqrt p-\arcsin\sqrt q\right|. \] The divergence of \(g_p\) at \(0\) and \(1\) is a coordinate effect; the boundary is at finite Fisher distance.
Volume and Jeffreys measure
A \(d\)-dimensional Riemannian metric determines the volume element \[ \mathrm{vol}_g(\mathrm{d}\theta) =\sqrt{\det I(\theta)}\,\mathrm{d}\theta^1\cdots\mathrm{d}\theta^d. \] The Jacobian factors cancel under reparametrization, so this defines an intrinsic measure. When normalized it is Jeffreys prior. Intrinsic does not mean automatically proper: location models on an unbounded parameter space usually have infinite Fisher volume.