Perturbation theory turns sensitivity into inequalities. It is the language behind condition numbers for roots, linear systems, eigenvalues, and least squares.
First-order perturbation expansion
Lemma. If \(\Phi\) is differentiable at \(x\), then \(\Phi(x+\Delta x)-\Phi(x)=D\Phi(x)\Delta x+o(\|\Delta x\|)\). The derivative is therefore the first-order amplifier of data error.
Linear systems
Proposition. For \(Ax=b\), perturbations satisfy approximately \(A\Delta x=\Delta b-\Delta A\,x\). Hence \(\|\Delta x\|/\|x\|\) is controlled by \(\kappa(A)(\|\Delta A\|/\|A\|+\|\Delta b\|/\|b\|)\), when the perturbations are small.
Simple eigenvalues
Theorem. If \(A v=\lambda v\) and \(w^\ast A=\lambda w^\ast\) with \(w^\ast v\ne0\), then a perturbation \(E\) changes the eigenvalue by \(\Delta\lambda=(w^\ast E v)/(w^\ast v)+o(\|E\|)\). Non-normal matrices may therefore have very sensitive eigenvalues.
Least squares
Proposition. The sensitivity of a least-squares solution depends on both the condition of \(A\) and the size of the residual. Problems with nearly rank-deficient design matrices require orthogonal factorizations or regularization rather than the normal equations alone.
Proof strategy
Proof. Perturbation bounds are usually obtained by linearizing the defining equation, solving the linearized equation, and then bounding the neglected higher-order term. The same template recurs in roots, eigenpairs, singular values, and discretized differential equations.
Linear systems
Theorem. Let \(Ax=b\), and suppose \((A+\Delta A)(x+\Delta x)=b+\Delta b\). If \(\|A^{-1}\Delta A\|<1\), then \[ \frac{\|\Delta x\|}{\|x\|} \le \frac{\kappa(A)} {1-\|A^{-1}\Delta A\|} \left( \frac{\|\Delta A\|}{\|A\|} +\frac{\|\Delta b\|}{\|b\|} \right). \]
Proof. Subtraction gives \[ (A+\Delta A)\Delta x=\Delta b-\Delta A x. \] The Neumann-series bound \[ \|(A+\Delta A)^{-1}\| \le\frac{\|A^{-1}\|}{1-\|A^{-1}\Delta A\|} \] and \(\|b\|\le\|A\|\|x\|\) yield the result.
Simple eigenvalues
Theorem (first-order eigenvalue perturbation). Let \(Ax=\lambda x\) and \(y^*A=\lambda y^*\), with \(y^*x\ne0\). For \(A(t)=A+tE\), a differentiable simple eigenvalue branch satisfies \[ \lambda'(0)=\frac{y^*Ex}{y^*x}. \]
Proof. Differentiate \(A(t)x(t)=\lambda(t)x(t)\) at zero and left-multiply by \(y^*\). Terms containing \(x'(0)\) cancel because \(y^*A=\lambda y^*\), leaving the formula.
The eigenvalue condition number is \[ \kappa(\lambda)=\frac{\|x\|\|y\|}{|y^*x|}. \] For a normal matrix, left and right eigenvectors coincide orthogonally and \(\kappa(\lambda)=1\). Nonnormal matrices can have extremely sensitive eigenvalues despite moderate entries.
Bauer--Fike and pseudospectra
Theorem (Bauer--Fike). If \(A=V\Lambda V^{-1}\), then every eigenvalue \(\mu\) of \(A+E\) satisfies \[ \min_{\lambda\in\sigma(A)}|\mu-\lambda| \le\kappa(V)\|E\|. \]
Proof. If \(\mu\notin\sigma(A)\), singularity of \(\mu I-A-E\) implies \[ 1\le\|(\mu I-A)^{-1}E\| \le\|V\|\|V^{-1}\|\|E\| \max_j|\mu-\lambda_j|^{-1}. \] Rearrange.
The \(\varepsilon\)-pseudospectrum \[ \sigma_\varepsilon(A) =\{z:\|(zI-A)^{-1}\|>\varepsilon^{-1}\} \] equals the union of spectra of \(A+E\) over \(\|E\|<\varepsilon\). It reveals nonnormal transient amplification invisible from eigenvalues alone.
Least-squares sensitivity
For full-column-rank \(A\), \(x=A^\dagger b\). Perturbations parallel to \(\operatorname{col}(A)\) behave like a linear solve, while perturbing \(A\) also rotates the column space. When the residual is large, the sensitivity can scale like \(\kappa(A)^2\). This is why normal equations, which explicitly square the condition number, are risky for ill-conditioned problems.