Gaussian laws are the unique class in which linear structure and probabilistic structure coincide completely. Means and covariance operators determine every finite-dimensional law; orthogonality becomes independence; conditioning is an affine projection; and a positive semidefinite kernel constructs an entire process.
Gaussian vectors and affine representations
Definition. A random vector \(X\in\mathbb R^d\) is Gaussian if \(t^\top X\) has a one-dimensional Gaussian distribution for every \(t\in\mathbb R^d\). Degenerate Gaussian variables, including constants, are allowed. Write \[ m=\mathbb EX,\qquad \Sigma=\mathbb E[(X-m)(X-m)^\top]. \]
Theorem. The following are equivalent:
- \(X\) is Gaussian with mean \(m\) and covariance \(\Sigma\);
- there are \(r\le d\), a matrix \(A\in\mathbb R^{d\times r}\), and \(Z\sim N_r(0,I_r)\) such that \[ X\stackrel d=m+AZ,\qquad AA^\top=\Sigma; \]
- the characteristic function is \[ \varphi_X(t) =\exp\!\left(it^\top m-\frac12t^\top\Sigma t\right). \]
Proof. If \(X=m+AZ\), every projection is \(t^\top m+(A^\top t)^\top Z\), hence Gaussian, and independence of the coordinates of \(Z\) gives \[ \mathbb Ee^{it^\top X} =e^{it^\top m}e^{-\|A^\top t\|^2/2} =e^{it^\top m-t^\top\Sigma t/2}. \] Conversely, diagonalize the nonnegative matrix \(\Sigma=Q\Lambda Q^\top\) and take \(A=Q\Lambda^{1/2}\), omitting zero eigenvalues. The vector \(m+AZ\) has the displayed characteristic function. A Gaussian vector has the same scalar characteristic function in every direction by the one-dimensional Gaussian formula, so uniqueness of characteristic functions identifies its law.
Remark. If \(\Sigma\) is singular, \(X\) is supported on the affine subspace \(m+\operatorname{Ran}(\Sigma)\). A Lebesgue density on all of \(\mathbb R^d\) exists only when \(\Sigma\) is positive definite.
Orthogonality and independence
Theorem. If \((X,Y)\) is jointly Gaussian, then \[ X\ \text{and}\ Y\text{ are independent} \quad\Longleftrightarrow\quad \operatorname{Cov}(X,Y)=0. \]
Proof. Independence always implies zero cross-covariance when second moments exist. Conversely, zero cross-covariance makes the joint covariance block diagonal: \[ \Sigma_{(X,Y)} = \begin{pmatrix} \Sigma_X&0\\0&\Sigma_Y \end{pmatrix}. \] The Gaussian characteristic function then factors: \[ \varphi_{(X,Y)}(s,t) = \varphi_X(s)\varphi_Y(t), \] which characterizes independence.
Example (uncorrelated is not generally independent). If \(Z\sim N(0,1)\), then \(\operatorname{Cov}(Z,Z^2-1)=\mathbb EZ^3=0\), but \(Z^2-1\) is determined by \(Z\). The theorem requires the pair itself to be jointly Gaussian; a nonlinear transform of a Gaussian variable usually is not Gaussian.
Gaussian conditioning
Theorem. Suppose \[ \begin{pmatrix}X\\Y\end{pmatrix} \sim N\!\left( \begin{pmatrix}m_X\\m_Y\end{pmatrix}, \begin{pmatrix} \Sigma_{XX}&\Sigma_{XY}\\ \Sigma_{YX}&\Sigma_{YY} \end{pmatrix}\right), \] and \(\Sigma_{YY}\) is invertible. Then \[ X\mid(Y=y) \sim N\!\left( m_X+\Sigma_{XY}\Sigma_{YY}^{-1}(y-m_Y), \Sigma_{XX}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX} \right). \]
Proof. Define the residual \[ R=X-m_X-\Sigma_{XY}\Sigma_{YY}^{-1}(Y-m_Y). \] It is jointly Gaussian with \(Y\), and \[ \operatorname{Cov}(R,Y) =\Sigma_{XY}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YY}=0. \] Hence \(R\) and \(Y\) are independent. Its covariance is the Schur complement \[ \operatorname{Cov}(R) =\Sigma_{XX}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}. \] Conditioning on \(Y=y\) leaves the independent residual law unchanged and turns the affine prediction term into the displayed mean.
The conditional mean is exactly the linear least-squares projection of \(X\) onto \(Y\). For Gaussian vectors, linear prediction is not merely optimal among linear predictors: it equals the full conditional expectation.
Gaussian concentration for linear functionals
Proposition. If \(X\sim N_d(m,\Sigma)\), then for every \(t\in\mathbb R^d\) and \(u>0\), \[ P\left(|t^\top(X-m)|\ge u\right) \le2\exp\!\left\{-\frac{u^2}{2t^\top\Sigma t}\right\}, \] with the natural interpretation when \(t^\top\Sigma t=0\).
Proof. The projection is \(N(0,v)\), \(v=t^\top\Sigma t\). For \(\lambda>0\), \[ P(t^\top(X-m)\ge u) \le e^{-\lambda u} \mathbb Ee^{\lambda t^\top(X-m)} =e^{-\lambda u+\lambda^2v/2}. \] Minimize at \(\lambda=u/v\), and apply the same bound to the negative tail.
Gaussian processes and canonical geometry
Definition. A process \(X=(X_t)_{t\in T}\) is Gaussian if every finite vector \((X_{t_1},\ldots,X_{t_n})\) is Gaussian. It is determined in finite-dimensional law by \[ m(t)=\mathbb EX_t,\qquad K(s,t)=\operatorname{Cov}(X_s,X_t). \]
Theorem. A function \(K:T\times T\to\mathbb R\) is the covariance kernel of a centered Gaussian process if and only if it is symmetric and positive semidefinite: \[ \sum_{i,j=1}^na_ia_jK(t_i,t_j)\ge0 \] for all finite choices. The process exists by Kolmogorov extension and is unique in finite-dimensional distribution.
Proof. Necessity follows from \[ \sum_{i,j}a_ia_jK(t_i,t_j) =\operatorname{Var}\!\left(\sum_ia_iX_{t_i}\right)\ge0. \] For sufficiency, each finite matrix \((K(t_i,t_j))\) defines a possibly singular Gaussian law. Gaussian marginalization makes these laws consistent, so the extension theorem applies.
Definition. The canonical pseudometric of a Gaussian process is \[ d_X(s,t) = \{\mathbb E(X_s-X_t)^2\}^{1/2} = \{K(s,s)+K(t,t)-2K(s,t)\}^{1/2}. \] It measures probabilistic rather than externally imposed distance between indices.
Examples.
- Brownian motion: \(K(s,t)=s\wedge t\), so \(d_B(s,t)=|s-t|^{1/2}\).
- Brownian bridge on \([0,1]\): \(K(s,t)=s\wedge t-st\).
- Stationary Ornstein-Uhlenbeck process: \(K(s,t)=(\sigma^2/2\theta)e^{-\theta|t-s|}\).