At a boundary point, the negative gradient need not vanish; it can be balanced by a normal force from the feasible set. Normal cones express this balance without selecting a particular inequality representation of the set.
Normals as polars of feasible motion
Definition. For a nonempty closed convex set \(C\subset\mathbb R^d\) and \(x\in C\), the normal cone is \[ N_C(x)= \{v:\langle v,y-x\rangle\le0\text{ for every }y\in C\}. \] Set \(N_C(x)=\varnothing\) when \(x\notin C\).
Normal-tangent polarity. For \(x\in C\), \[ N_C(x)=T_C(x)^\circ :=\{v:\langle v,w\rangle\le0\ \forall w\in T_C(x)\}. \] Moreover \(N_C(x)=\partial\iota_C(x)\).
Proof. If \(v\in N_C(x)\), then \(\langle v,t(y-x)\rangle\le0\) for every \(t\ge0,y\in C\); continuity extends the inequality to the closure \(T_C(x)\). Conversely, \(y-x\in T_C(x)\) for every \(y\in C\), so membership in the polar implies the defining normal inequality. Finally, \(v\in\partial\iota_C(x)\) exactly when \(\iota_C(y)\ge\iota_C(x)+\langle v,y-x\rangle\) for all \(y\); only \(y\in C\) imposes a nontrivial condition, and that condition is normality.
Half-spaces and orthants. For \(C=\{x:a^\top x\le b\}\), \[ N_C(x)= \begin{cases} \{0\},&a^\top x<b,\\ \{\lambda a:\lambda\ge0\},&a^\top x=b. \end{cases} \] For \(C=\mathbb R_+^d\), a vector \(v\in N_C(x)\) satisfies \(v_i=0\) when \(x_i>0\) and \(v_i\le0\) when \(x_i=0\). These componentwise relations are complementary slackness in geometric form.
Constrained Fermat rule
Variational inequality. Let \(C\) be nonempty, closed, and convex, and let \(f\) be differentiable and convex. For \(x^\star\in C\), the following are equivalent:
\[ \begin{aligned} &(i)\quad x^\star\in\operatorname*{arg\,min}_{x\in C}f(x),\\ &(ii)\quad \langle\nabla f(x^\star),x-x^\star\rangle\ge0 \quad(x\in C),\\ &(iii)\quad 0\in\nabla f(x^\star)+N_C(x^\star). \end{aligned} \]
Proof. If \(x^\star\) minimizes, apply the one-dimensional necessary condition to \(t\mapsto f(x^\star+t(x-x^\star))\) at \(t=0\), proving (ii). Convexity gives \[ f(x)\ge f(x^\star)+ \langle\nabla f(x^\star),x-x^\star\rangle, \] so (ii) implies global optimality. Finally, (ii) says precisely that \(-\nabla f(x^\star)\in N_C(x^\star)\).
Projection as a fixed-point equation
Projected optimality. Under the preceding assumptions, \(x^\star\) is optimal if and only if, for one equivalently every \(\eta>0\), \[ x^\star=P_C\!\left(x^\star-\eta\nabla f(x^\star)\right). \]
Proof. The projection characterization says \(p=P_Cz\) exactly when \(\langle z-p,x-p\rangle\le0\) for all \(x\in C\). Substitute \(p=x^\star\) and \(z=x^\star-\eta\nabla f(x^\star)\), then divide by \(-\eta<0\). The resulting inequality is the variational inequality above.
This equivalence explains projected gradient descent: \[ x_{k+1}=P_C(x_k-\eta_k\nabla f(x_k)). \] The algorithm seeks a fixed point of the exact optimality map. Projection is not an after-the-fact correction; it is the resolvent \((I+N_C)^{-1}\) of the normal-cone operator.