An inner product adds metric geometry to vector spaces: lengths, angles, orthogonality, best approximation, and adjoints. Many later decompositions are algebraic consequences of orthogonality.
Inner products and norms
Definition. An inner product on a real vector space is a map \(\langle\cdot, \cdot\rangle:V\times V\to\mathbb R\) that is bilinear, symmetric, and positive definite. On a complex vector space we use the convention that it is linear in the first variable, conjugate-linear in the second, Hermitian symmetric, and positive definite. The associated norm is \[ \|v\|=\sqrt{\langle v,v\rangle}. \]
Example. With this convention, the standard inner product on \(F^n\) is \(\langle x,y\rangle=y^*x\). On \(C[a,b]\), one uses \(\langle f,g\rangle=\int_a^b f(t)\overline{g(t)}\,\mathrm{d}t\). On \(M_{m,n}(F)\), the Frobenius inner product is \(\langle A,B\rangle=\operatorname{tr}(B^*A)\).
Cauchy-Schwarz and triangle inequality
Theorem. For any vectors \(u,v\), \[ |\langle u,v\rangle|\le \|u\|\|v\|. \] Consequently, \(\|u+v\|\le \|u\|+ \|v\|\).
Proof. If \(v=0\) the result is clear. Otherwise choose \(a=\langle u,v\rangle/\langle v,v\rangle\). Positivity of \(\|u-av\|^2\) gives \[ 0\le \|u\|^2-\frac{|\langle u,v\rangle|^2}{\|v\|^2}, \] which is Cauchy-Schwarz. The triangle inequality follows by expanding \(\|u+v\|^2\) and bounding the cross term by Cauchy-Schwarz.
Orthogonal complements
Definition. For \(S\subset V\), the orthogonal complement is \[ S^\perp=\{v\in V:\langle v,s\rangle=0\text{ for all }s\in S\}. \] It is always a subspace, even when \(S\) is not.
Theorem. If \(U\) is a finite-dimensional subspace of an inner product space \(V\), then \[ V=U\oplus U^\perp \] whenever \(V\) is finite-dimensional. More generally, every \(v\in V\) has a unique best approximation from \(U\), namely its orthogonal projection.
Gram-Schmidt orthogonalization
Theorem. Every linearly independent list \((v_1, \ldots,v_n)\) in an inner product space can be converted into an orthonormal list \((q_1, \ldots,q_n)\) with the same nested spans: \[ \operatorname{span}(q_1, \ldots,q_k)=\operatorname{span}(v_1, \ldots,v_k). \]
Calculation. With the linear-in-the-first convention, the projection of \(v_k\) onto the unit vector \(q_j\) is \[ \operatorname{proj}_{q_j}(v_k)=\langle v_k,q_j\rangle q_j. \] Gram-Schmidt removes all previous orthonormal components and then normalizes: \[ w_k=v_k-\sum_{j=1}^{k-1}\langle v_k,q_j\rangle q_j, \qquad q_k=\frac{w_k}{\|w_k\|}. \] Independence ensures \(w_k\ne0\). If one uses the opposite convention, with the inner product linear in the second variable, the coefficient becomes \(\langle q_j,v_k\rangle\).
For instance, in \(\mathbb R^2\), take \(v_1=(1,1)\) and \(v_2=(1,0)\). Then \[ \begin{aligned} q_1&=\frac{1}{\sqrt{2}}(1,1),\\ \langle v_2,q_1\rangle&=\frac{1}{\sqrt{2}},\\ w_2 &=v_2-\langle v_2,q_1\rangle q_1\\ &=(1,0)-\frac{1}{2}(1,1)\\ &=\frac{1}{2}(1,-1),\\ q_2&=\frac{w_2}{\|w_2\|} =\frac{1}{\sqrt{2}}(1,-1). \end{aligned} \] This gives an orthonormal list \((q_1,q_2)\) with the same nested spans as \((v_1,v_2)\).
Orthogonal matrices and unitary matrices
Proposition. A square real matrix \(Q\) is orthogonal, or a complex matrix is unitary, if \(Q^*Q=I\). Its columns form an orthonormal basis, \(Q^{-1}=Q^*\), and \(\|Qx\|=\|x\|\) for every \(x\).
Remark. Orthogonality is the algebraic form of no interference. When subspaces are orthogonal, dimensions add cleanly, projections are stable, and computations avoid the amplification that appears in nearly dependent coordinate systems.
Gram-Schmidt as triangular factorization
Theorem. For linearly independent \(v_1,\ldots,v_n\), define recursively \[ u_k=v_k-\sum_{j<k}\langle v_k,q_j\rangle q_j,\qquad q_k=\frac{u_k}{\|u_k\|}. \] Then \((q_1,\ldots,q_n)\) is orthonormal and \[ \operatorname{span}(q_1,\ldots,q_k) =\operatorname{span}(v_1,\ldots,v_k) \] for every \(k\).
Proof. Induct on \(k\). For \(i<k\), \[ \langle u_k,q_i\rangle =\langle v_k,q_i\rangle -\sum_{j<k}\langle v_k,q_j\rangle\langle q_j,q_i\rangle=0. \] If \(u_k=0\), then \(v_k\) lies in the span of its predecessors, contradicting independence. The recursive formula expresses \(q_k\) in the old span and \(v_k\) in the new span, proving equality of the flags.
If \(V=[v_1\ \cdots\ v_n]\) and \(Q=[q_1\ \cdots\ q_n]\), the coefficients \(r_{jk}=\langle q_j,v_k\rangle\) form an upper triangular matrix with positive diagonal and \[ V=QR. \] Thus Gram-Schmidt is the geometric content of QR factorization. Modified Gram-Schmidt computes the same exact factorization but is more stable in floating-point arithmetic because it updates the residual after each projection.
Bessel and Parseval identities
Proposition. For an orthonormal family \((e_j)_{j=1}^m\), \[ \sum_{j=1}^m|\langle x,e_j\rangle|^2\le\|x\|^2. \] Equality holds exactly when \(x\) lies in their span.
Proof. The residual \(r=x-\sum_j\langle x,e_j\rangle e_j\) is orthogonal to every \(e_j\). Pythagoras gives \[ \|x\|^2=\|r\|^2+\sum_j|\langle x,e_j\rangle|^2. \]