The probability simplex is the simplest full statistical manifold. Many information-geometric formulas become explicit there and reveal the meaning of Fisher geometry.
Interior simplex
The natural manifold is the strictly positive simplex, where support does not change.
Definition. The \(n\)-category probability simplex is \[ \Delta_n^\circ=\{p\in\mathbb R^n:p_i>0,\ \sum_i p_i=1\}. \] Its tangent space at \(p\) is \(T_p\Delta_n^\circ=\{u\in\mathbb R^n:\sum_i u_i=0\}\).
Proposition. The Fisher metric on the simplex is \[ g_p(u,v)=\sum_i\frac{u_iv_i}{p_i}. \]
Proof. For coordinate perturbation \(u\), the score component is \(u_i/p_i\) on category \(i\). Taking the expectation under \(p\) gives \(\sum_i p_i(u_i/p_i)(v_i/p_i)=\sum_i u_iv_i/p_i\).
Square-root sphere
The square-root map turns the simplex into a piece of a sphere.
Theorem. The map \(p\mapsto 2\sqrt p\) is an isometric embedding from \((\Delta_n^\circ,g)\) into the sphere of radius \(2\).
Proof. The differential sends \(u\) to \(u_i/\sqrt{p_i}\). The Euclidean inner product of two such differentials is \(\sum_i u_iv_i/p_i\), the Fisher metric. The image has squared norm \(4\sum_i p_i=4\).
Mixture and exponential coordinates
The simplex carries two natural affine coordinate systems.
Definition. Mixture coordinates use probabilities \(p_i\) subject to \(\sum p_i=1\). Exponential coordinates use log odds, for example \(\theta_i=\log(p_i/p_n)\) for \(i<n\). These coordinates are dual in the dually flat geometry of the simplex.
Remark. Mixture geodesics are straight lines in probabilities. Exponential geodesics are straight lines in log probabilities, followed by normalization.
Boundary behavior
The Fisher metric blows up near the boundary in probability coordinates.
Remark. As \(p_i\downarrow0\), the term \(u_i^2/p_i\) becomes large. This reflects the statistical singularity of changing a nearly impossible category by a fixed absolute amount.
Tangent and cotangent descriptions
For \[ \Delta_{m-1}^{\circ} =\{p\in\mathbb R^m:p_i>0,\ \mathbf1^\top p=1\}, \] the tangent space is \(T_p\Delta=\{u:\mathbf1^\top u=0\}\). A tangent vector may instead be represented by score coordinates \(a_i=u_i/p_i\), satisfying \(\sum_i p_i a_i=0\). In these representations, \[ g_p(u,v)=\sum_i\frac{u_iv_i}{p_i} =\sum_i p_i a_i b_i. \]
Mixture, exponential, and square-root coordinates
Three systems expose three kinds of straightness:
- Mixture coordinates use \(p_1,\ldots,p_{m-1}\), with affine segments \(p(t)=(1-t)p+tq\).
- Exponential coordinates \(\theta_i=\log(p_i/p_m)\) have affine segments \(p_i(t)\propto p_i^{1-t}q_i^t\).
- Square-root coordinates \(s_i=2\sqrt{p_i}\) place the simplex on a sphere; great-circle arcs are Levi-Civita geodesics.
Proposition. In mixture coordinates \(\eta_i=p_i\), \(i<m\), the Fisher matrix and inverse are \[ g_{ij}=\frac{\delta_{ij}}{p_i}+\frac1{p_m}, \qquad g^{ij}=p_i\delta_{ij}-p_ip_j. \]
Proof. The coordinate tangent \(\partial_i\) changes \(p_i\) by \(1\) and \(p_m\) by \(-1\), so its score is \(\mathbf1_{\{X=i\}}/p_i-\mathbf1_{\{X=m\}}/p_m\). Taking products gives the first matrix. Direct multiplication gives the inverse.
Boundary strata and metric completion
The open simplex is a manifold, but its closure is stratified by supports. Fisher coefficients diverge as \(p_i\downarrow0\), while square-root distance to a face remains finite. Thus the metric completion is the closed spherical orthant. Exponential coordinates send the boundary to infinity, explaining why an exponential-family maximum-likelihood estimate may fail to exist as a finite natural parameter even when a limiting distribution exists.