Markov morphisms formalize stochastic transformations of observations. Monotonicity says that processing data cannot increase distinguishability between distributions.
Markov kernels as maps
A Markov kernel maps probability laws on one space to probability laws on another.
Definition. Given a Markov kernel \(K(\mathrm{d}y\mid x)\), the pushforward of \(P\) is \[ (KP)(B)=\int K(B\mid x)P(\mathrm{d}x). \] This includes deterministic statistics as the special case \(K(B\mid x)=\mathbf 1\{T(x)\in B\}\).
Remark. Markov morphisms are the categorical maps of statistical experiments: they represent randomization, coarse-graining, and observation channels.
Data processing for KL
KL divergence decreases under Markov kernels.
Theorem. For probability laws \(P,Q\) and a Markov kernel \(K\), \[ D_{\mathrm{KL}}(KP\|KQ)\le D_{\mathrm{KL}}(P\|Q). \]
Proof. In the dominated finite case, write the likelihood ratio after applying \(K\) as a conditional expectation of the original likelihood ratio under the joint law. Jensen's inequality for \(u\log u\) gives the inequality. The general case follows by standard approximation and disintegration.
Monotone metrics
A metric is information monotone if stochastic maps cannot increase squared lengths of tangent vectors.
Definition. A Riemannian metric \(g\) on probability models is monotone if for every Markov kernel \(K\), \[ g_{KP}(Ku,Ku)\le g_P(u,u) \] for tangent vectors \(u\) that can be pushed forward by \(K\).
Proposition. The Fisher metric is monotone under Markov kernels.
Proof. Represent a tangent vector by a score \(s\). After applying the channel, the pushed-forward score is the conditional expectation \(\mathbb E(s\mid Y)\). Therefore \[ \mathbb E\{\mathbb E(s\mid Y)^2\}\le \mathbb E(s^2) \] by Jensen or conditional-variance decomposition.
Sufficiency and equality
Equality in monotonicity is tied to preservation of the relevant likelihood ratios.
Remark. For sufficient statistics, no information is lost and Fisher lengths along the model are preserved. This is the geometric form of the factorization principle.
Statistical maps and score contraction
A Markov kernel \(K(\mathrm{d}y\mid x)\) sends \(P\) to \(PK(B)=\int K(B\mid x)P(\mathrm{d}x)\). If \(X\sim P_\theta\) and \(Y\mid X\sim K(\cdot\mid X)\), where \(K\) is parameter independent, then the induced score is \[ \dot\ell_\theta^Y(Y) =E_\theta\{\dot\ell_\theta^X(X)\mid Y\}. \]
Theorem (infinitesimal data processing). For every tangent direction \(v\), \[ g_{PK}(K_*v,K_*v)\le g_P(v,v). \] Equivalently, \(I_Y(\theta)\preceq I_X(\theta)\).
Proof. The score identity and conditional Jensen give \[ E\!\left[E\{\ell_v(X)\mid Y\}^2\right] \le E\{\ell_v(X)^2\}. \] Polarization supplies the bilinear contraction. Geometrically, a Markov map sends a tangent score to its orthogonal projection onto the subspace observable through \(Y\).
Equality and sufficiency
Proposition. Equality in Fisher contraction for a direction \(v\) holds if and only if \(\ell_v(X)\) is measurable with respect to \(Y\). Equality for every direction holds when \(Y\) is sufficient; under standard domination and connectedness conditions, the converse is locally valid.
Proof. Conditional variance gives \[ E\ell_v^2 =E\{E(\ell_v\mid Y)^2\}+E\{\operatorname{Var}(\ell_v\mid Y)\}. \] Thus equality is equivalent to zero conditional variance. If \(Y\) is sufficient, the conditional law of \(X\) given \(Y\) is independent of \(\theta\); differentiating the factorization shows that every score is a function of \(Y\).
Congruent embeddings and coarse graining
On finite alphabets, a deterministic statistic combines cells, while a congruent embedding splits each cell into subcells with parameter-independent probabilities. The latter has a Markov left inverse and therefore preserves all information. A monotone metric contracts under every stochastic map and is preserved by congruent embeddings. These operations form the algebraic core of the Cencov characterization.
Example (binning). For a multinomial model with cell score \(s_i\), merging cells in a block \(B\) replaces the score by \[ \bar s_B=\frac{\sum_{i\in B}p_is_i}{\sum_{i\in B}p_i}. \] The information loss inside that block is \[ \sum_{i\in B}p_i(s_i-\bar s_B)^2. \] Binning is lossless exactly when the score is constant in every block.