Longitudinal causal inference studies treatment sequences that respond to evolving covariates. Time-varying covariates can simultaneously predict future treatment, predict outcome, and be affected by prior treatment, creating treatment-confounder feedback.
Histories and counterfactual regimes
Definition. At time \(t\), let \[ H_t=(\overline L_t,\overline A_{t-1}) \] be observed history. A deterministic dynamic regime \(d=(d_0,\ldots,d_K)\) sets \(A_t=d_t(H_t)\). The counterfactual outcome under this regime is \(Y^d\), and its value is \(V(d)=\mathbb EY^d\).
Sequential exchangeability and positivity
Definition. Sequential exchangeability requires \[ Y^d\perp\!\!\!\perp A_t\mid H_t \] for every compatible regime \(d\) and time \(t\). Sequential positivity requires \[ P\{A_t=d_t(H_t)\mid H_t\}>0 \] on histories that can occur under \(d\).
Positivity is regime-specific. A deterministic regime may be unsupported even when a nearby stochastic intervention is well supported.
Iterated conditional expectation
Theorem. Define recursively \[ Q_{K+1}=Y,\qquad Q_t(h_t)= \mathbb E\{Q_{t+1}(H_{t+1})\mid H_t=h_t,A_t=d_t(h_t)\}. \] Under consistency, sequential exchangeability, and positivity, \[ V(d)=\mathbb E\{Q_0(L_0)\}. \]
Proof. At the final decision time, exchangeability and consistency identify the conditional mean of the future counterfactual under the prescribed action. Apply iterated expectation backward one time point at a time. Each recursion integrates over the covariate transition distribution generated under previous regime actions.
Inverse-probability weights for regimes
Theorem. For a fixed regime \(\overline a\), \[ \mathbb EY^{\overline a} = \mathbb E\left[ \frac{\mathbf1(\overline A_K=\overline a_K)Y} {\prod_{t=0}^K p(A_t=a_t\mid H_t)} \right]. \]
Proof. Condition successively on \(H_K,H_{K-1},\ldots,H_0\). At each step, the indicator divided by its conditional treatment probability integrates to one, while exchangeability and consistency identify the counterfactual continuation. The telescoping expectations yield \(\mathbb EY^{\overline a}\).
Marginal structural models
Definition. A marginal structural model parameterizes \[ \mathbb EY^{\overline a}=m(\overline a;\beta). \] It is fitted in a pseudo-population weighted by inverse probabilities of observed treatment histories, often stabilized by numerator probabilities with reduced conditioning sets.
The model is marginal over time-varying confounders. Including those confounders as ordinary regression covariates would condition away part of prior treatment effects; weighting instead balances them across subsequent treatment choices.
Structural nested mean models and g-estimation
A structural nested mean model parameterizes the causal contrast from “blipping down” one treatment at a time. G-estimation chooses the parameter that makes the blipped-down outcome conditionally independent of current treatment given history. This avoids modeling the full outcome distribution and is particularly useful when treatment effects are naturally expressed as sequential increments.
Optimal dynamic regimes
Theorem. For finite horizon and binary actions, define \[ Q_t(h,a)= \mathbb E\{V_{t+1}(H_{t+1})\mid H_t=h,A_t=a\}, \qquad V_t(h)=\max_aQ_t(h,a), \] with \(V_{K+1}=Y\). Then the optimal regime is \[ d_t^*(h)\in\arg\max_aQ_t(h,a). \]
Proof. Backward induction. At the final time, maximizing conditional expected outcome is optimal. Assuming \(V_{t+1}\) is the optimal continuation value, action \(a\) at time \(t\) yields conditional value \(Q_t(h,a)\); choosing its maximizer is therefore optimal from that history.
This is the causal analogue of Bellman's principle. Identification requires sequential exchangeability; statistical reinforcement learning often instead relies on an environment interaction or Markov decision process assumption.
Q-learning and nonregularity
Q-learning estimates the backward regressions and chooses the maximizing action. If two actions have equal value with positive probability, the max operator is nondifferentiable and treatment-rule parameters can be nonregular. Standard bootstrap or Wald inference may fail; margin conditions, smoothing, or value-focused inference are alternatives.
Stochastic and incremental policies
Definition. An incremental propensity intervention with odds multiplier \(\delta>0\) sets \[ q_\delta(1\mid h)= \frac{\delta e(h)} {\delta e(h)+1-e(h)}. \] It shifts observed treatment odds rather than assigning treatment where it was never observed.
Such policies can be identified without requiring both treatment levels at every history in the same strong form as deterministic regimes, because the intervention remains absolutely continuous with respect to the observed mechanism.
Censoring as a treatment process
Loss to follow-up can be represented by a censoring indicator at each time. Under conditional independent censoring, inverse probability of censoring weights multiply treatment weights. The product can be highly variable; separate diagnostics for treatment and censoring overlap are needed.
Cross-fitting longitudinal scores
Longitudinal TMLE and sequentially doubly robust estimators combine backward outcome regressions with treatment/censoring mechanisms. Their remainders contain sums of products of nuisance errors at each time. Cross-fitting permits machine learning, but long horizons magnify both positivity problems and accumulated nuisance error.