Non-Parametric Tests for Treatment Effect Heterogeneity in Randomized Experiments and Observational Studies
Skip to main content
eScholarship
Open Access Publications from the University of California

UC Irvine

UC Irvine Electronic Theses and Dissertations bannerUC Irvine

Non-Parametric Tests for Treatment Effect Heterogeneity in Randomized Experiments and Observational Studies

Abstract

Comprehensively assessing the effect of a treatment usually includes two objectives, estimating the average treatment effect across the whole target population and evaluating variability of the treatment effect across different subpopulations in an effort to provide more precise treatment recommendations. A common way to identify treatment effect heterogeneity is to split the sample into several strata based on one or more baseline covariates which may be relevant to the effect of treatment, and then compare the localized or stratum-specific treatment effects across those strata. Parametric approaches have been proposed to compare average treatment effects across several strata. One approach is testing interactions between treatment indicator and group indicators in linear regressions (Allison, 1977), and another approach is the likelihood ratio test proposed by Gail and Simon(1985). Both of them require parametric assumptions of outcome distributions. When the parametric assumptions fail, the test may be invalid or the power may be negatively impacted. Thus there is a need for non-parametric tests that can better adapt to various outcome distributions. Randomized experiments are considered to be the gold standard for assessing treatment effects, as all baseline covariates are expected to be well balanced in treatment groups after randomization. However, randomized experiments are not always feasible due to various obstacles, e.g., ethical concerns and high expense. Therefore researchers turn to observational studies. Not only can they avoid the obstacles faced by randomized experiments, but because there are often fewer exclusionary characteristics, the results of observational studies may generalize better to the target population. A main challenge of observational studies is controlling confounding variables. There is a considerable literature on causal inference in observational studies that has been developed targeting this challenge. Many of the proposed procedures balance the observed variables and then rely on the unconfoundedness assumption, i.e., all confounding variables are observed. This assumption is not only strong but also unverifiable. The violation of this assumption can invalidate causal conclusions. A variety of approaches to assessing the sensitivity of causal conclusions to violations of the unconfoundedness assumption have been proposed. By assessing the extent of the assumption violation required to change the conclusion and evaluating the possibility of such a violation based on domain knowledge, it is possible to provide more reliable conclusions. In this dissertation, we describe three contributions we have made to the goal of comprehensively evaluating the effectiveness of treatments. The first two contributions are non-parametric U-statistic-based tests examining the variability of treatment effects across different subpopulations. The first procedure can be appropriately applied in cases, like randomized experiments, where all baseline covariates are well balanced within each stratum; the second procedure adjusts unbalanced confounding variables using propensity scores. Compared to their parametric counterparts, likelihood ratio tests, our non-parametric tests are more powerful when the distributions of study outcomes depart substantially from the distributions assumed by likelihood ratio tests. The third contribution is a sensitivity analysis that addresses the concern of possible violation of the unconfoundedness assumption for the adjusted Mann-Whitney test, a non-parametric test that evaluates the existence of treatment effects in observational studies.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View