A pointwise law of large numbers controls each fixed function. Glivenko-Cantelli theory controls the supremum over a class and is therefore the basic consistency mechanism for extremum estimators, data-dependent choices, and uniform risk approximation.
Uniform laws and a counterexample
Definition. A measurable function class \(\mathcal F\) is \(P\)-Glivenko-Cantelli if \[ \|P_n-P\|_{\mathcal F} =\sup_{f\in\mathcal F}|P_nf-Pf| \to0 \] in outer probability. It is strong \(P\)-Glivenko-Cantelli if convergence holds outer almost surely.
Counterexample (pointwise does not imply uniform). Let \(P\) be nonatomic and let \(\mathcal F\) contain indicators of all finite subsets of \(\mathcal X\). For each fixed finite \(A\), \[ P_n(A)\to P(A)=0. \] But the random finite set \(A_n=\{X_1,\ldots,X_n\}\) belongs to the class and satisfies \[ P_n(A_n)=1,\qquad P(A_n)=0. \] Therefore \(\|P_n-P\|_{\mathcal F}=1\) for every \(n\).
The supremum effectively selects a function after seeing the sample. Complexity restrictions prevent this unrestricted interpolation.
Finite bracketing
Definition. A bracket \[ [l,u]=\{f:l\le f\le u\} \] has \(L^1(P)\)-size \(P(u-l)\). The bracketing number \[ N_{[]}(\varepsilon,\mathcal F,L^1(P)) \] is the smallest number of brackets of size below \(\varepsilon\) needed to cover \(\mathcal F\).
Finite-bracket lemma. Suppose \[ \mathcal F\subseteq\bigcup_{j=1}^N[l_j,u_j], \qquad P(u_j-l_j)\le\varepsilon, \] and all bracket endpoints are integrable. Then \[ \limsup_{n\to\infty} \|P_n-P\|_{\mathcal F} \le\varepsilon \] almost surely.
Proof. If \(f\in[l_j,u_j]\), then \[ (P_n-P)f \le(P_n-P)u_j+P(u_j-f) \le(P_n-P)u_j+P(u_j-l_j), \] and similarly \[ (P_n-P)f \ge(P_n-P)l_j-P(u_j-l_j). \] Hence \[ |(P_n-P)f| \le \max\{|(P_n-P)l_j|,|(P_n-P)u_j|\} +\varepsilon. \] Take the supremum over \(f\), then the maximum over finitely many endpoints. The strong law sends that maximum to zero.
Bracketing Glivenko-Cantelli theorem. If \[ N_{[]}(\varepsilon,\mathcal F,L^1(P))\lt\infty \quad\text{for every }\varepsilon>0, \] with integrable bracket endpoints and appropriate measurability, then \(\mathcal F\) is strong \(P\)-Glivenko-Cantelli.
Proof. Apply the finite-bracket lemma with \(\varepsilon=1/m\) for each integer \(m\). Intersect the resulting countably many probability-one events. On this intersection, \[ \limsup_n\|P_n-P\|_{\mathcal F}\le1/m \] for every \(m\), so the limsup is zero.
A compact Lipschitz-parametric criterion
Parametric uniform law. Let \(\Theta\subseteq\mathbb R^d\) be compact. Suppose \[ |f_\theta(x)-f_\eta(x)| \le L(x)\|\theta-\eta\|, \qquad PL\lt\infty, \] and \(P|f_{\theta_0}|\lt\infty\) for one \(\theta_0\). Then \[ \sup_{\theta\in\Theta} |(P_n-P)f_\theta|\to0 \quad\text{almost surely}. \]
Proof. Cover compact \(\Theta\) by finitely many Euclidean balls of radius \(\delta\) with centers \(\theta_j\). If \(\theta\) belongs to the \(j\)-th ball, then \[ |(P_n-P)(f_\theta-f_{\theta_j})| \le(P_n+P)|f_\theta-f_{\theta_j}| \le\delta(P_n+P)L. \] Consequently, \[ \sup_\theta|(P_n-P)f_\theta| \le\max_j|(P_n-P)f_{\theta_j}| +\delta(P_n+P)L. \] The finite maximum tends to zero, and \(P_nL\to PL\). Take the limsup, then let \(\delta\downarrow0\).
Closure properties
Permanence of Glivenko-Cantelli classes. If \(\mathcal F,\mathcal G\) are \(P\)-Glivenko-Cantelli with integrable envelopes, then:
- every subclass of \(\mathcal F\) is Glivenko-Cantelli;
- \(\mathcal F\cup\mathcal G\) and \(\mathcal F+\mathcal G\) are Glivenko-Cantelli;
- if the functions are uniformly bounded, \(\mathcal F\mathcal G=\{fg:f\in\mathcal F,g\in\mathcal G\}\) is Glivenko-Cantelli;
- if \(\varphi\) is uniformly Lipschitz on the common range, \(\varphi\circ\mathcal F\) is Glivenko-Cantelli.
Proof. Subclasses and unions follow from supremum inequalities. For sums, \[ \|(P_n-P)(\mathcal F+\mathcal G)\| \le\|P_n-P\|_{\mathcal F} +\|P_n-P\|_{\mathcal G}. \] For products, write \[ fg-f'g' =f(g-g')+g'(f-f') \] and use finite brackets or total boundedness with the uniform bounds. The Lipschitz composition follows from \[ |\varphi(f)-\varphi(g)| \le L|f-g| \] and the same bracketing argument.
Consistency of extremum estimators
Argmax consistency. Let \[ \hat\theta_n\in\Theta \] satisfy \[ P_nm_{\hat\theta_n} \ge\sup_{\theta\in\Theta}P_nm_\theta-o_p(1). \] Suppose \(\{m_\theta:\theta\in\Theta\}\) is \(P\)-Glivenko-Cantelli and, for every \(\varepsilon>0\), \[ Pm_{\theta_0}> \sup_{d(\theta,\theta_0)\ge\varepsilon} Pm_\theta. \] Then \(\hat\theta_n\to_p\theta_0\).
Proof. Let the population separation outside the \(\varepsilon\)-ball be \(3\eta>0\). On the event \[ \sup_\theta|(P_n-P)m_\theta|<\eta \] and with optimization error below \(\eta\), every point outside the ball has empirical criterion at least \(\eta\) below the criterion at \(\theta_0\). Therefore the approximate maximizer lies inside the ball. The Glivenko-Cantelli property makes the event's probability tend to one.
Triangular and random classes
A fixed-class theorem does not automatically cover \(\mathcal F_n\) that grows with \(n\), nor a random class selected from the data. One needs entropy bounds uniform in \(n\), localization, sample splitting, or a deterministic class containing the random functions with high probability. This distinction becomes central for sieves, high-dimensional models, and estimated nuisance functions.