UC Davis

Background

The theoretical conditions under which causal estimates can be derived from observational data are challenging to achieve in the real world. Applied examples can help elucidate the practical limitations of methods to estimate randomized-controlled trial effects from observational data.

Methods

We used six methods with varying design and analytic features to compare the 5-year risk of incident myocardial infarction among statin users and non-users, and used non-cardiovascular mortality as a negative control outcome. Design features included restriction to a statin-eligible population and new users only; analytic features included multivariable adjustment and propensity score matching.

Results

We used data from 5294 participants in the Cardiovascular Health Study from 1989 to 2004. For non-cardiovascular mortality, most methods produced protective estimates with confidence intervals that crossed the null. The hazard ratio (HR) was 0.92, 95% confidence interval: 0.58, 1.46 using propensity score matching among eligible new users. For myocardial infarction, all estimates were strongly protective; the propensity score-matched analysis among eligible new users resulted in a HR of 0.55 (0.29, 1.05)-a much stronger association than observed in randomized controlled trials.

Conclusions

In designs that compare active treatment with non-treated participants to evaluate effectiveness, methods to address bias in observational data may be limited in real-world settings by residual bias.