Parameter-dependent integrals are controlled by convergence theorems. Continuity and differentiability under the integral sign require uniform integrable bounds.
Continuity under the integral sign
Dominated convergence gives the basic continuity theorem.
Theorem. Let \(f:\Theta\times X\to\mathbb R\) be measurable in \(x\), continuous in \(\theta\) at \(\theta_0\) for almost every \(x\), and satisfy \(|f(\theta,x)|\le g(x)\in L^1\) near \(\theta_0\). Then \(F(\theta)=\int f(\theta,x)\,\mathrm{d}\mu(x)\) is continuous at \(\theta_0\).
Proof. If \(\theta_n\to\theta_0\), then \(f(\theta_n,x)\to f(\theta_0,x)\) almost everywhere and is dominated by \(g\). Dominated convergence gives \(F(\theta_n)\to F(\theta_0)\), which is sequential continuity.
Differentiation under the integral sign
A dominated derivative permits exchanging derivative and integral.
Theorem. Suppose \(\theta\mapsto f(\theta,x)\) is differentiable on a neighborhood of \(\theta_0\) for almost every \(x\), \(f(\theta_0,\cdot)\in L^1\), and \[ |\partial_\theta f(\theta,x)|\le g(x) \] there for some \(g\in L^1\). Then \(F(\theta)=\int f(\theta,x)\,\mathrm{d}\mu(x)\) is differentiable at \(\theta_0\) and \[ F'(\theta_0)=\int \partial_\theta f(\theta_0,x)\,\mathrm{d}\mu(x). \]
Proof. For \(h\ne0\), set \[ q_h(x)=\frac{f(\theta_0+h,x)-f(\theta_0,x)}h. \] Pointwise differentiability gives \(q_h(x)\to\partial_\theta f(\theta_0,x)\). By the one-dimensional mean value theorem, for almost every \(x\), \[ |q_h(x)|\le\sup_{\theta\text{ between }\theta_0\text{ and }\theta_0+h} |\partial_\theta f(\theta,x)| \le g(x). \] In particular, \(f(\theta_0+h,\cdot)\) is integrable, and dominated convergence gives \[ \frac{F(\theta_0+h)-F(\theta_0)}h =\int q_h\,\mathrm{d}\mu \longrightarrow\int\partial_\theta f(\theta_0,x)\,\mathrm{d}\mu(x). \]
Vector parameters
Gradients are handled coordinatewise or by linearization.
Proposition. If \(\nabla_\theta f(\theta,x)\) exists near \(\theta_0\), is pointwise continuous at \(\theta_0\), and \(\|\nabla_\theta f(\theta,x)\|\le g(x)\in L^1\), then \(\nabla F(\theta_0)=\int\nabla_\theta f(\theta_0,x)\,\mathrm{d}\mu(x)\).
Proof. For almost every \(x\), the fundamental theorem along the segment gives \[ f(\theta_0+h,x)-f(\theta_0,x) =\int_0^1\nabla_\theta f(\theta_0+th,x)^\top h\,\mathrm{d}t. \] Subtract \(\nabla_\theta f(\theta_0,x)^\top h\), divide by \(\|h\|\), and bound the absolute value by \[ \int_0^1\|\nabla_\theta f(\theta_0+th,x)-\nabla_\theta f(\theta_0,x)\|\,\mathrm{d}t \le2g(x). \] Pointwise continuity makes this tend to zero, and dominated convergence proves a Fréchet remainder \(o(\|h\|)\) after integration. The derivative is the stated integral vector.
Second derivatives and Hessians
Theorem. Let \(\Theta\subset\mathbb R^p\) be open. Suppose \(f(\cdot,x)\) is twice continuously differentiable almost everywhere and, on every compact \(K\subset\Theta\), \[ \sup_{\theta\in K}\|\nabla_\theta^2f(\theta,x)\| \le g_K(x) \] for some \(g_K\in L^1\). If \(f(\theta_0,\cdot)\) and \(\nabla_\theta f(\theta_0,\cdot)\) are integrable at one point, then \[ \nabla F(\theta)=\int\nabla_\theta f(\theta,x)\,\mathrm{d}\mu(x), \qquad \nabla^2F(\theta)=\int\nabla_\theta^2f(\theta,x)\,\mathrm{d}\mu(x). \]
Proof. On a compact line segment, the Hessian envelope and the mean value theorem dominate gradient increments by an integrable function. The vector-parameter theorem therefore differentiates \(F\) once. Applying the same theorem to each component of \(\nabla_\theta f\), now dominated through the Hessian envelope, gives the second identity and continuity of the integrated Hessian.
Leibniz rule with moving endpoints
Boundary motion contributes endpoint terms.
Theorem. If \(F(\theta)=\int_{a(\theta)}^{b(\theta)} f(\theta,x)\,\mathrm{d}x\), with \(a,b\) differentiable and \(f,\partial_\theta f\) continuous, then \[ F'(\theta)=f(\theta,b(\theta))b'(\theta)-f(\theta,a(\theta))a'(\theta)+\int_{a(\theta)}^{b(\theta)}\partial_\theta f(\theta,x)\,\mathrm{d}x. \]
Proof. Rewrite the integral over a fixed reference interval by an affine change of variables from \([0,1]\) to \([a(\theta),b(\theta)]\). Differentiate under the integral sign using the previous theorem and simplify the derivative of the transformed integrand.
Scores and derivatives of expectations
When the measure itself depends on the parameter, both the integrand and the density contribute.
Remark. If \(P_\theta\) has density \(p_\theta\) relative to \(\lambda\), and differentiation is justified, then \[ \frac{\mathrm{d}}{\mathrm{d}\theta}\mathbb E_\theta[h_\theta(X)] =\mathbb E_\theta[\partial_\theta h_\theta(X)] +\mathbb E_\theta[h_\theta(X)s_\theta(X)], \qquad s_\theta=\partial_\theta\log p_\theta. \] Taking \(h_\theta\equiv1\) gives \(\mathbb E_\theta s_\theta=0\). These identities require a common dominating measure and an integrable derivative envelope; they are not formal consequences of notation.