Maximum likelihood is efficient when consistency, local quadratic curvature, and identifiability all hold. Maximization itself supplies none of these properties. Support changes, unbounded likelihoods, boundaries, mixtures, incidental parameters, and misspecification each break a different step of the regular proof.
Approximate maximizers
Definition. An approximate MLE satisfies \[ \ell_n(\hat\theta_n) \geq\sup_{\theta\in\Theta}\ell_n(\theta)-o_p(1). \] For root-\(n\) asymptotics, an \(o_p(1)\) log-likelihood error is enough because local log-likelihood differences are order one.
The regular MLE theorem
Theorem. Suppose:
- \(\hat\theta_n\to_p\theta_0\);
- the model is uniformly LAN in a neighborhood of \(\theta_0\) with nonsingular \(I_{\theta_0}\);
- the local maximizer is tight and the likelihood outside root-\(n\) neighborhoods is uniformly lower;
- the approximate maximization error is \(o_p(1)\).
Then \[ \sqrt n(\hat\theta_n-\theta_0) = I_{\theta_0}^{-1}\Delta_n+o_p(1) \rightsquigarrow N_p(0,I_{\theta_0}^{-1}). \]
Proof. Consistency localizes the estimator. The separation condition strengthens localization to root-\(n\) tightness. On bounded local sets, uniform LAN makes the likelihood an \(o_p(1)\) perturbation of \[ h\mapsto h^\top\Delta_n-\tfrac12h^\top I_{\theta_0}h, \] whose unique maximizer is \(I_{\theta_0}^{-1}\Delta_n\). The argmax theorem gives the expansion, and the central-sequence CLT gives the limit. \(\square\)
Neyman–Scott incidental parameters
Example. Observe, for \(i=1,\ldots,n\), \[ X_{i1},X_{i2}\overset{\mathrm{ind}}{\sim} N(\mu_i,\sigma^2), \] with common \(\sigma^2\) and one nuisance mean \(\mu_i\) per pair. The MLEs are \[ \hat\mu_i=\frac{X_{i1}+X_{i2}}2, \qquad \hat\sigma_{\mathrm{MLE}}^2 = \frac1{2n}\sum_{i=1}^n \sum_{j=1}^2(X_{ij}-\hat\mu_i)^2. \] Since \[ \sum_{j=1}^2(X_{ij}-\hat\mu_i)^2 =\frac12(X_{i1}-X_{i2})^2, \] and \(E(X_{i1}-X_{i2})^2=2\sigma^2\), \[ \hat\sigma_{\mathrm{MLE}}^2\to_p\frac{\sigma^2}{2}. \] The MLE is inconsistent because nuisance dimension grows with sample size.
The unbiased within-pair estimator divides the residual sum by \(n\), not \(2n\). Profiling out an increasing number of nuisance parameters consumed a nonvanishing fraction of the information.
Unbounded mixture likelihoods
Example (Gaussian mixture). In a finite Gaussian mixture with unknown component variances, place one component mean at an observed point and let its variance tend to zero. Its density at that point diverges like \(\sigma^{-1}\), while the remaining component fits the other observations. The likelihood is unbounded and an ordinary MLE does not exist.
Constraints, penalties, or priors preventing variance collapse are structural modifications of the decision problem, not mere numerical fixes.
Separation in logistic regression
Example. If a vector \(b\) satisfies \[ x_i^\top b>0\quad\text{for all }Y_i=1, \qquad x_i^\top b<0\quad\text{for all }Y_i=0, \] then along \(\beta=tb\), \(t\to\infty\), every fitted probability tends to the observed label and the logistic likelihood increases toward its supremum. No finite MLE exists. Firth correction or a proper prior restores finite estimates by altering the criterion.
Boundary parameters
Example. Suppose an unconstrained estimator satisfies \[ \sqrt n(\tilde\theta_n-\theta_0)\rightsquigarrow N(0,\sigma^2), \] but the parameter space is \([0,\infty)\) and \(\theta_0=0\). The constrained MLE is approximately \[ \hat\theta_n=\max(0,\tilde\theta_n), \] so \[ \sqrt n\,\hat\theta_n \rightsquigarrow\max(0,\sigma Z), \] with an atom of mass \(1/2\) at zero. Likelihood-ratio statistics have chi-bar-square rather than ordinary chi-square limits.
Support-dependent models
Example (uniform endpoint). For \(X_i\sim U[0,\theta]\), the MLE is \(X_{(n)}\). It converges at rate \(n\): \[ n\left(1-\frac{X_{(n)}}{\theta}\right) \rightsquigarrow\operatorname{Exp}(1). \] The model is not DQM because the support changes with \(\theta\), so root-\(n\) LAN theory does not apply.
Misspecified likelihood
Theorem. Under regular misspecification, the MLE converges to the KL projection \(\theta^\dagger\) and \[ \sqrt n(\hat\theta_n-\theta^\dagger) \rightsquigarrow N(0,A^{-1}BA^{-\top}), \] where \[ A=-P\ddot\ell_{\theta^\dagger}, \qquad B=P(\dot\ell_{\theta^\dagger}\dot\ell_{\theta^\dagger}^\top). \]
Under correct specification \(A=B=I\); under misspecification they differ. The inverse observed Hessian alone estimates \(A^{-1}\), not the sandwich covariance.
Decision-theoretic diagnosis
Before invoking “MLE efficiency,” verify existence, identification, consistency, the correct local rate, likelihood-ratio convergence, nuisance dimension, and the loss under which optimality is claimed. The experiment determines the asymptotics; maximization is only a construction within it.