The strong law of large numbers is a pathwise theorem: outside one null set, every sufficiently late sample average is close to its mean. The proof below isolates the three mechanisms that recur throughout almost-sure asymptotics: a maximal inequality, convergence of a random series, and truncation of rare large observations.
Kolmogorov's maximal inequality
Theorem (Kolmogorov maximal inequality). Let \(X_1,\ldots,X_n\) be independent, centered, square-integrable random variables, and write \(S_k=\sum_{j\le k}X_j\). Then for every \(\lambda>0\), \[ P\!\left(\max_{1\le k\le n}|S_k|\ge\lambda\right) \le\frac{\operatorname{Var}(S_n)}{\lambda^2} =\frac{\sum_{j=1}^n\operatorname{Var}(X_j)}{\lambda^2}. \]
Proof. Let \[ A_k=\{|S_1|<\lambda,\ldots,|S_{k-1}|<\lambda,\ |S_k|\ge\lambda\}. \] The \(A_k\) are disjoint and their union is the event on the left. Since \(A_k\in\sigma(X_1,\ldots,X_k)\), the increment \(S_n-S_k\) is independent of \(1_{A_k}S_k\) and has mean zero. Hence \[ \mathbb E[S_n^2 1_{A_k}] =\mathbb E[(S_k+(S_n-S_k))^2 1_{A_k}] \ge\mathbb E[S_k^2 1_{A_k}] \ge\lambda^2P(A_k). \] Summing over \(k\) gives \[ \operatorname{Var}(S_n)=\mathbb ES_n^2 \ge\lambda^2\sum_{k=1}^nP(A_k), \] which is the desired bound.
Convergence of independent series
Proposition (Kolmogorov convergence criterion). If \(X_n\) are independent and centered with \[ \sum_{n=1}^{\infty}\operatorname{Var}(X_n)<\infty, \] then \(\sum_nX_n\) converges almost surely and in \(L^2\).
Proof. For \(m<n\), independence gives \[ \mathbb E|S_n-S_m|^2=\sum_{k=m+1}^n\operatorname{Var}(X_k)\to0, \] so \(S_n\) is Cauchy in \(L^2\) and converges in \(L^2\) to some \(S\). To obtain almost-sure convergence, choose integers \(n_j\uparrow\infty\) such that \[ \sum_{k>n_j}\operatorname{Var}(X_k)\le 2^{-3j}. \] Kolmogorov's inequality gives \[ P\!\left(\max_{n_j<r\le n_{j+1}}|S_r-S_{n_j}|>2^{-j}\right) \le 2^{2j}\sum_{k=n_j+1}^{n_{j+1}}\operatorname{Var}(X_k) \le2^{-j}. \] Borel-Cantelli shows that the displayed block oscillations tend to zero almost surely. Also \[ P(|S_{n_j}-S|>2^{-j})\le 2^{2j}\mathbb E|S_{n_j}-S|^2\le2^{-j}, \] so another application of Borel-Cantelli gives \(S_{n_j}\to S\) almost surely. Combining the subsequence convergence with the vanishing block oscillations proves \(S_n\to S\) almost surely.
Lemma (Kronecker). If \(b_n>0\), \(b_n\uparrow\infty\), and the numerical series \(\sum_nx_n/b_n\) converges, then \[ \frac1{b_n}\sum_{k=1}^nx_k\longrightarrow0. \]
Proof. Put \(s_n=\sum_{k=1}^nx_k/b_k\), so \(s_n\to s\). Summation by parts yields \[ \sum_{k=1}^nx_k =b_ns_n-\sum_{k=1}^{n-1}s_k(b_{k+1}-b_k). \] Divide by \(b_n\), insert \(s_k=s+(s_k-s)\), and note that the terms involving \(s\) cancel. Given \(\varepsilon>0\), choose \(N\) such that \(|s_k-s|<\varepsilon\) for \(k\ge N\). The finitely many terms \(k<N\), divided by \(b_n\), tend to zero, while the remaining weighted sum is bounded by \(\varepsilon(b_n-b_N)/b_n\le\varepsilon\).
Corollary. If \(X_n\) are independent, centered, square-integrable, and \[ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2}<\infty, \] then \[ \frac1n\sum_{k=1}^nX_k\longrightarrow0\qquad\text{almost surely}. \]
Proof. Apply the convergence criterion to \(X_n/n\), then apply Kronecker's lemma with \(b_n=n\).
The i.i.d. strong law
Theorem (Kolmogorov strong law). If \(X_1,X_2,\ldots\) are i.i.d. and \(\mathbb E|X_1|<\infty\), then \[ \frac1n\sum_{k=1}^nX_k\longrightarrow\mathbb EX_1 \qquad\text{almost surely}. \]
Proof. Define the observation-dependent truncations \[ Y_n=X_n1_{\{|X_n|\le n\}}. \] Since \[ \sum_{n=1}^{\infty}P(X_n\ne Y_n) =\sum_{n=1}^{\infty}P(|X_1|>n) \le\mathbb E|X_1|<\infty, \] Borel-Cantelli implies \(X_n=Y_n\) for all sufficiently large \(n\), almost surely. Thus replacing \(X_n\) by \(Y_n\) changes partial sums by an eventually constant random quantity, whose quotient by \(n\) tends to zero. It remains to control the centered variables \(Z_n=Y_n-\mathbb EY_n\). By Tonelli's theorem, \[ \begin{aligned} \sum_{n=1}^{\infty}\frac{\operatorname{Var}(Y_n)}{n^2} &\le \sum_{n=1}^{\infty}\frac{\mathbb E[X_1^2 1_{\{|X_1|\le n\}}]}{n^2}\\ &=\mathbb E\!\left[X_1^2\sum_{n\ge |X_1|\vee1}\frac1{n^2}\right] \le C\mathbb E|X_1|<\infty, \end{aligned} \] because \(\sum_{n\ge m}n^{-2}\le C/m\). The corollary therefore gives \[ \frac1n\sum_{k=1}^n(Y_k-\mathbb EY_k)\to0 \quad\text{almost surely}. \] Finally, dominated convergence gives \(\mathbb EY_n\to\mathbb EX_1\), and Ces?ro averaging gives \[ \frac1n\sum_{k=1}^n\mathbb EY_k\to\mathbb EX_1. \] Combining the centered term, the deterministic bias, and the finitely many discarded observations proves the theorem.
Sharpness of integrability
Proposition (converse). If \(X_n\) are i.i.d. and \(S_n/n\) converges almost surely to a finite random variable, then \(\mathbb E|X_1|<\infty\). In that case the limit is the constant \(\mathbb EX_1\).
Proof. Convergence of \(S_n/n\) implies \[ \frac{X_n}{n}=\frac{S_n}{n}-\frac{n-1}{n}\frac{S_{n-1}}{n-1}\longrightarrow0 \quad\text{almost surely}. \] Hence, for every \(\varepsilon>0\), the independent events \(\{|X_n|>\varepsilon n\}\) occur only finitely often. The converse Borel-Cantelli lemma forces \[ \sum_{n=1}^{\infty}P(|X_1|>\varepsilon n)<\infty. \] The tail-sum criterion then yields \(\mathbb E|X_1|<\infty\). The strong law already proved identifies the limit as \(\mathbb EX_1\).
Example (Cauchy observations). If \(X_i\) are standard Cauchy, stability gives \(S_n/n\stackrel d=X_1\) for every \(n\). Therefore the averages cannot converge in probability, and hence cannot converge almost surely, to a constant. This is not a minor technical failure: the first moment required by the strong law does not exist.
For dependent sequences, strong laws are often consequences of ergodicity rather than independence; see the ergodic theorem. For Banach-valued or function-indexed averages, geometry and uniform laws enter through functional analysis and empirical-process theory.