Center for Effective Global Action
Inference without randomization or ignorability: A stability controlled quasi-experiment on the prevention of tuberculosis
- Author(s): Hazlett, Chad
- Maokola, Werner
- Wulf, David Ami
- et al.
When determining the eﬀectiveness of a new treatment, randomized trials are not always possible or desirable. The stability-controlled quasi-experiment (SCQE) (Hazlett, 2019) is an observational approach that replaces the usual “no-unobserved confounding” assumption with one on the change in non-treatment outcome between successive cohorts, or the “baseline trend.” We extend this method to allow variance estimation and inference, and apply it for the ﬁrst time by examining whether isoniazid preventive therapy (IPT) reduced tuberculosis (TB) incidence among 26,715 HIV patients in Tanzania. After IPT became available in the clinics we studied, a non-random 25% of patients received it. Within a year, fewer than 1% of patients on IPT developed TB, compared to 16% of the untreated. Regression adjustment using available covari-ates produces an equally large and highly signiﬁcant estimate of -15 percentage point (pp) [95%CI: -16.6, -13.7]. While those estimates may generate conﬁdence in IPT’s eﬀectiveness, they cannot eliminate confounding. By contrast, SCQE reveals that the average treatment eﬀect on the treated must be small and indistinguishable from zero, if we assume the baseline trend was ﬂat over the study period. Rather, to argue that IPT was beneﬁcial requires claiming that the (non-treatment) incidence rate rose by at least 0.5 pp per year. This is plausible, but far from certain. The SCQE approach has broad applicability and will sometimes lead to deﬁnitive claims of eﬀectiveness. In this case, it usefully aids in protecting against over-conﬁdence in claims that IPT was eﬀective.