A Riemannian metric measures lengths, but a connection tells us how to differentiate vector fields and compare tangent vectors at nearby points. Information geometry studies families of affine connections adapted to statistical models.
Affine connection
A connection is a rule for differentiating vector fields along vector fields.
Definition. An affine connection \(\nabla\) on a manifold assigns to vector fields \(X,Y\) another vector field \(\nabla_XY\), linear in \(X\), linear over constants in \(Y\), and satisfying the Leibniz rule \[ \nabla_X(fY)=X(f)Y+f\nabla_XY. \]
Remark. In coordinates, \(\nabla_{\partial_i}\partial_j=\Gamma^k_{ij}\partial_k\). The functions \(\Gamma^k_{ij}\) are Christoffel symbols, but they are not tensor components.
Torsion
Most statistical connections used here are torsion-free.
Definition. The torsion tensor is \[ T^\nabla(X,Y)=\nabla_XY-\nabla_YX-[X,Y]. \] A connection is torsion-free if \(T^\nabla=0\).
Proposition. In a coordinate frame, torsion-free means \(\Gamma^k_{ij}=\Gamma^k_{ji}\).
Proof. Coordinate vector fields commute: \([\partial_i,\partial_j]=0\). Thus \(T(\partial_i,\partial_j)=(\Gamma^k_{ij}-\Gamma^k_{ji})\partial_k\). Vanishing torsion is equivalent to equality of the lower indices.
Covariant derivative of tensors
Connections extend from vector fields to tensors by compatibility with contraction and product rules.
Definition. For a covector field \(\omega=\omega_j\mathrm{d}\theta^j\), \[ (\nabla_i\omega)_j=\partial_i\omega_j-\Gamma^k_{ij}\omega_k. \] For a vector field \(Y=Y^k\partial_k\), \[ (\nabla_iY)^k=\partial_iY^k+\Gamma^k_{ij}Y^j. \]
Remark. These formulas ensure that \(\nabla_i(\omega_jY^j)=\partial_i(\omega_jY^j)\), so scalar contractions differentiate ordinarily.
Why connections enter statistics
Different affine structures correspond to different ways of declaring curves of distributions straight.
Remark. Mixture families are flat for the mixture connection; exponential families are flat for the exponential connection. Their duality is one of the central ideas of information geometry.
Why ordinary differentiation is not intrinsic
For a vector field \(Y=Y^j\partial_j\), the coordinate derivatives \(\partial_iY^j\) do not transform tensorially because the basis \(\partial_j\) itself varies. An affine connection \(\nabla\) corrects this variation. In coordinates, \[ \nabla_{\partial_i}\partial_j=\Gamma^k{}_{ij}\partial_k, \qquad (\nabla_XY)^k=X^i(\partial_iY^k+\Gamma^k{}_{ij}Y^j). \]
Definition. An affine connection is a map \((X,Y)\mapsto\nabla_XY\) satisfying \[ \nabla_{fX+gZ}Y=f\nabla_XY+g\nabla_ZY,\qquad \nabla_X(fY)=X(f)Y+f\nabla_XY. \] It is \(C^\infty\)-linear in its first argument and obeys a Leibniz rule in its second.
Proposition. Under coordinates \(\eta^a\), the coefficients transform as \[ \widetilde\Gamma^c{}_{ab} =\frac{\partial\eta^c}{\partial\theta^k} \frac{\partial\theta^i}{\partial\eta^a} \frac{\partial\theta^j}{\partial\eta^b}\Gamma^k{}_{ij} +\frac{\partial\eta^c}{\partial\theta^k} \frac{\partial^2\theta^k}{\partial\eta^a\partial\eta^b}. \] Thus Christoffel symbols are not tensor components.
Proof. Expand \(\nabla_{\partial_{\eta^a}}\partial_{\eta^b}\) using \(\partial_{\eta^a}=(\partial\theta^i/\partial\eta^a)\partial_{\theta^i}\) and the Leibniz rule. The derivative of \(\partial\theta^j/\partial\eta^b\) produces the inhomogeneous second-derivative term. Converting the output basis to \(\partial_{\eta^c}\) gives the formula.
Torsion and curvature
The torsion and curvature are genuine tensors: \[ T(X,Y)=\nabla_XY-\nabla_YX-[X,Y], \] \[ R(X,Y)Z=\nabla_X\nabla_YZ-\nabla_Y\nabla_XZ-\nabla_{[X,Y]}Z. \] In a coordinate frame, \[ T^k{}_{ij}=\Gamma^k{}_{ij}-\Gamma^k{}_{ji}, \] \[ R^\ell{}_{kij} =\partial_i\Gamma^\ell{}_{jk}-\partial_j\Gamma^\ell{}_{ik} +\Gamma^m{}_{jk}\Gamma^\ell{}_{im} -\Gamma^m{}_{ik}\Gamma^\ell{}_{jm}. \] A torsion-free flat connection admits local affine coordinates in which every \(\Gamma^k{}_{ij}\) vanishes.
Covariant derivatives of tensors
For a covector \(\omega=\omega_j\mathrm{d}\theta^j\) and a \((0,2)\)-tensor \(A\), \[ \nabla_i\omega_j=\partial_i\omega_j-\Gamma^k{}_{ij}\omega_k, \] \[ \nabla_iA_{jk} =\partial_iA_{jk}-\Gamma^\ell{}_{ij}A_{\ell k} -\Gamma^\ell{}_{ik}A_{j\ell}. \] The opposite signs for upper and lower indices ensure that contraction commutes with \(\nabla\).
Theorem (fundamental theorem of Riemannian geometry). Every Riemannian metric \(g\) has a unique torsion-free connection satisfying \(\nabla g=0\). Its coefficients are \[ \Gamma^{(0)k}{}_{ij} =\frac12g^{k\ell} (\partial_i g_{j\ell}+\partial_j g_{i\ell}-\partial_\ell g_{ij}). \]
Information geometry retains this Levi--Civita connection but also uses metric-dual pairs \((\nabla,\nabla^*)\). Their departures from the midpoint \(\nabla^{(0)}\) encode third-order, asymmetric information absent from the Fisher metric alone.