A divergence is not a metric, but its derivatives near the diagonal generate a metric and affine connections. This is the main mechanism by which information geometry extracts local geometry from global contrast functions.
Divergence functions
A divergence vanishes on the diagonal and is positive away from it, at least locally.
Definition. A smooth divergence on a manifold \(M\) is a function \(D:M\times M\to[0,\infty)\) such that \(D(p,q)=0\) iff \(p=q\), and the first derivatives vanish on the diagonal: \[ \partial_iD(p,q)|_{q=p}=0,\qquad \partial_{j'}D(p,q)|_{q=p}=0. \]
Remark. The prime on an index indicates differentiation in the second argument. This convention is essential when deriving two dual connections from one function.
Metric from second derivatives
The Hessian of a divergence at the diagonal gives the Riemannian metric.
Theorem. Define \[ g_{ij}(p)=-\partial_i\partial_{j'}D(p,q)\big|_{q=p}. \] Under the usual positivity assumptions, \(g\) is a Riemannian metric.
Proof. Because the first derivatives vanish on the diagonal, the second-order Taylor expansion of \(D(p,p+\mathrm{d}p)\) is quadratic. The mixed Hessian gives the bilinear form controlling this quadratic term. Nonnegativity of \(D\) near the diagonal forces this form to be positive semidefinite; strict local separation gives positive definiteness.
Dual connections from third derivatives
Third derivatives distinguish the two arguments of the divergence and produce dual affine connections.
Definition. The divergence-induced connections are commonly defined by \[ \Gamma_{ijk}=-\partial_i\partial_j\partial_{k'}D|_{\mathrm{diag}}, \qquad \Gamma^*_{ijk}=-\partial_{i'}\partial_{j'}\partial_kD|_{\mathrm{diag}}. \]
Proposition. The two connections are dual with respect to the metric generated by the same divergence.
Proof. Differentiate the identity \(g_{ij}=-\partial_i\partial_{j'}D|_{\mathrm{diag}}\) along a coordinate direction. The derivative hits both arguments on the diagonal and splits into the two third-derivative expressions above. This is exactly the coordinate form of metric duality.
Local versus global information
Different divergences can generate the same local metric while differing globally.
Remark. KL, Hellinger, and many \(f\)-divergences all generate Fisher information locally, but their global projection behavior and asymmetry differ. Local geometry and global optimization should not be conflated.
Contrast functions and diagonal derivatives
Definition. A smooth divergence on a manifold \(\mathcal M\) is a function \(\mathcal D:\mathcal M\times\mathcal M\to[0,\infty)\) such that \(\mathcal D(p,q)=0\) if and only if \(p=q\). It need not be symmetric or obey the triangle inequality. Write \(\partial_i\) for differentiation in the first argument and \(\partial_{j'}\) in the second.
Lemma. If \(\mathcal D\) is \(C^2\) near the diagonal, then \[ \partial_i\mathcal D|_{q=p} =\partial_{j'}\mathcal D|_{q=p}=0, \] and \[ g_{ij} =-\partial_i\partial_{j'}\mathcal D|_{q=p} =\partial_i\partial_j\mathcal D|_{q=p} =\partial_{i'}\partial_{j'}\mathcal D|_{q=p} \] defines a positive semidefinite bilinear form.
Proof. For fixed \(p\), \(q=p\) is a minimum of \(q\mapsto\mathcal D(p,q)\), and similarly in the first argument, so first derivatives vanish. Differentiate the identity \(\mathcal D(\theta,\theta)=0\) once and twice. The second derivative gives \[ 0=\partial_i\partial_j\mathcal D +\partial_i\partial_{j'}\mathcal D +\partial_{i'}\partial_j\mathcal D +\partial_{i'}\partial_{j'}\mathcal D \] on the diagonal; symmetry of same-argument Hessians and the first-derivative identities yield the equal formulas. Nonnegativity along \(\mathcal D(\theta,\theta+tv)\) makes the quadratic coefficient nonnegative.
Local equivalence and global inequivalence
For \(q=\theta+h\), \[ \mathcal D(\theta,\theta+h) =\frac12g_{ij}(\theta)h^ih^j+O(\|h\|^3). \] Thus divergences with the same Hessian induce the same local lengths and first-order efficiency theory. Their cubic and higher derivatives may differ, leading to different dual connections, projections, tail penalties, and optimization behavior.
Example. On positive scalars, \[ D_1(p,q)=p\log(p/q)-p+q,\qquad D_2(p,q)=q\log(q/p)-q+p \] both induce \(g_p(u,v)=uv/p\), but their directed projections onto a nonconvex set can be entirely different. A local metric cannot recover a divergence globally.
Third derivatives and dual connections
Under a fixed sign convention, Eguchi's construction sets \[ \Gamma_{ijk} =-\partial_i\partial_j\partial_{k'}\mathcal D|, \qquad \Gamma^*_{ijk} =-\partial_{i'}\partial_{j'}\partial_k\mathcal D|, \] where the vertical bar denotes the diagonal. Swapping arguments exchanges the connections. Their average is Levi--Civita and their difference is encoded by a symmetric cubic tensor. Since authors vary signs and argument order, every calculation should begin by stating this convention rather than relying on the labels “primal” and “dual.”
Degenerate contrasts
A nonnegative contrast may have a singular Hessian. For example, \(\mathcal D(\theta,\eta)=(\theta-\eta)^4\) separates points but induces the zero quadratic form. To generate a Riemannian geometry one must assume nondegeneracy of the mixed Hessian. At boundaries or singular statistical models, even KL divergence can induce a degenerate metric, so the appropriate object is then stratified or sub-Riemannian rather than regular Riemannian geometry.