In several variables, differentiability means approximation by a linear map, not merely the existence of partial derivatives.
Fr?chet derivative
The derivative is the linear map that captures first-order behavior.
Definition. A map \(f:U\subset\mathbb R^d\to\mathbb R^m\) is differentiable at \(x\) if there is a linear map \(Df(x)\) such that \[ f(x+h)=f(x)+Df(x)h+r(h), \qquad \frac{\|r(h)\|}{\|h\|}\longrightarrow0. \]
Proposition. The derivative \(Df(x)\), when it exists, is unique; differentiability implies continuity at \(x\).
Proof. If linear maps \(A,B\) both satisfy the definition, then for every fixed \(v\), \[ \|(A-B)v\| =\frac{\|(A-B)(tv)\|}{|t|} =o(1) \] as \(t\to0\), hence \((A-B)v=0\). Thus \(A=B\). Moreover, \[ \|f(x+h)-f(x)\| \le\|Df(x)\|\,\|h\|+o(\|h\|)\longrightarrow0. \]
Proposition. If \(f\) is differentiable at \(x\), then all directional derivatives exist and equal \(Df(x)v\).
Proof. Substitute \(h=tv\) into the Fr?chet expansion and divide by \(t\). The error \(o(|t|\|v\|)/t\) tends to zero as \(t\to0\).
Partial derivatives are not enough
Partial derivatives see only coordinate directions.
Remark. Existence of all partial derivatives at a point does not imply even continuity. For \[ f(x,y)= \begin{cases} \dfrac{xy}{x^2+y^2},&(x,y)\ne(0,0),\\ 0,&(x,y)=(0,0), \end{cases} \] both partial derivatives at the origin are zero, but \(f(t,t)=1/2\). Even existence of every directional derivative is insufficient: for \[ g(x,y)= \begin{cases} \dfrac{x^3}{x^2+y^2},&(x,y)\ne(0,0),\\ 0,&(x,y)=(0,0), \end{cases} \] the directional derivative at \(0\) in direction \(v=(a,b)\) is \(a^3/(a^2+b^2)\), which is not linear in \(v\) and therefore cannot be a Fréchet derivative.
Theorem. If all first partial derivatives of \(f\) exist in a neighborhood of \(x\) and are continuous at \(x\), then \(f\) is differentiable at \(x\).
Proof. It is enough to treat scalar \(f\), component by component. Put \[ x^{(j)}=x+(h_1,\ldots,h_j,0,\ldots,0). \] Then \[ f(x+h)-f(x)=\sum_{j=1}^d\bigl(f(x^{(j)})-f(x^{(j-1)})\bigr). \] The one-dimensional mean value theorem gives intermediate points \(\xi_j\) on the \(j\)-th coordinate segment such that \[ f(x^{(j)})-f(x^{(j-1)}) =\partial_jf(\xi_j)h_j. \] Because \(\|\xi_j-x\|\le\|h\|\), continuity at \(x\) gives \[ \max_j|\partial_jf(\xi_j)-\partial_jf(x)|\longrightarrow0. \] Hence the difference from \(\sum_j\partial_jf(x)h_j\) is bounded by \[ \max_j|\partial_jf(\xi_j)-\partial_jf(x)|\sum_j|h_j| \le\sqrt d\,o(1)\|h\|=o(\|h\|). \]
Chain rule
Linear approximations compose by matrix multiplication.
Theorem. If \(g\) is differentiable at \(x\) and \(f\) is differentiable at \(g(x)\), then \(f\circ g\) is differentiable at \(x\) and \(D(f\circ g)(x)=Df(g(x))Dg(x)\).
Proof. Use \(g(x+h)=g(x)+Dg(x)h+r(h)\), where \(\|r(h)\|=o(\|h\|)\), and apply the expansion of \(f\) at \(g(x)\). Since \(\|Dg(x)h+r(h)\|=O(\|h\|)\), the remainder from \(f\) is \(o(\|h\|)\).
Mean value inequality
Derivative bounds imply global Lipschitz bounds on convex domains.
Proposition. If \(U\) is convex and \(\|Df(z)\|\le M\) for all \(z\in U\), then \(\|f(x)-f(y)\|\le M\|x-y\|\) for \(x,y\in U\).
Proof. Define \(\gamma(t)=f(y+t(x-y))\). Then \(\gamma'(t)=Df(y+t(x-y))(x-y)\), so \(\|\gamma'(t)\|\le M\|x-y\|\). Integrating \(\gamma'\) from \(0\) to \(1\) gives the estimate.
Remark. There is generally no vector-valued identity \[ f(x)-f(y)=Df(z)(x-y) \] for one intermediate point \(z\). The scalar mean value theorem applies after pairing with a linear functional, while the norm inequality above follows by integrating along the segment.
Second-order expansion
Hessians encode local curvature.
Theorem. If \(f\in C^2\) near \(x\), then \[ f(x+h)=f(x)+\nabla f(x)^\top h+\frac12 h^\top \nabla^2 f(x)h+o(\|h\|^2). \]
Proof. Apply the one-dimensional Taylor theorem to \(\phi(t)=f(x+th)\) at \(t=0\). Then \(\phi'(0)=\nabla f(x)^\top h\) and \(\phi''(0)=h^\top\nabla^2 f(x)h\), while continuity of the Hessian gives the Peano remainder.
Theorem (symmetry of mixed partials). If \(\partial_i\partial_j f\) and \(\partial_j\partial_i f\) exist on a neighborhood of \(x\) and are continuous at \(x\), then \[ \partial_i\partial_jf(x)=\partial_j\partial_if(x). \] Thus the Hessian of a \(C^2\) scalar function is symmetric.
Proof. Restrict to the two relevant coordinates and consider the rectangular increment \[ \Delta(h,k)=f(x+he_i+ke_j)-f(x+he_i)-f(x+ke_j)+f(x). \] Applying the one-dimensional mean value theorem first in direction \(e_i\) and then \(e_j\) gives \[ \Delta(h,k)=hk\,\partial_j\partial_if(\xi_{h,k}) \] for a point \(\xi_{h,k}\) in the rectangle. Reversing the order gives \[ \Delta(h,k)=hk\,\partial_i\partial_jf(\eta_{h,k}). \] As \(h,k\to0\), both intermediate points approach \(x\); continuity and division by \(hk\) yield equality.
Proposition (second-order optimality). If \(x\) is an interior local minimizer of a twice differentiable \(f:\mathbb R^d\to\mathbb R\), then \[ \nabla f(x)=0,\qquad h^\top\nabla^2f(x)h\ge0\quad\text{for every }h. \] Conversely, if \(\nabla f(x)=0\) and \(\nabla^2f(x)\) is positive definite, then \(x\) is a strict local minimizer.
Proof. For fixed \(h\), the one-dimensional function \(t\mapsto f(x+th)\) has a local minimum at \(0\), so its first derivative vanishes and its second derivative is nonnegative. This gives the necessary conditions. If the Hessian is positive definite, compactness of the unit sphere gives \(\lambda>0\) with \(h^\top\nabla^2f(x)h\ge\lambda\|h\|^2\). The second-order expansion then yields \[ f(x+h)-f(x)\ge\frac{\lambda}{4}\|h\|^2>0 \] for all sufficiently small nonzero \(h\).