Skip to main content
eScholarship
Open Access Publications from the University of California

UC San Diego

UC San Diego Previously Published Works bannerUC San Diego

‘Optimism bias’ in contemporary national clinical trial network phase III trials: are we improving?

Abstract

Background

Previous studies have found that overestimating treatment effects (i.e. 'optimism bias') leads to underpowered clinical trials. The prevalence of 'optimism bias' in contemporary National Clinical Trials Network (NCTN) cancer clinical trials is unknown.

Methods

We conducted a systematic review of NCTN phase III randomized trials published from January 2007 to January 2017. We compared the hypothesized versus observed treatment effects in each trial, and examined whether trial-related factors were correlated with the study results. We also reviewed the methods of each protocol to assess whether a rationale for the hypothesized effect size was provided.

Results

We identified 161 clinical trials, of which 130 were eligible for analysis. Original protocols could not be located for 8 trials (5.0%). Twenty-eight trials (21.5%) observed a statistically significant difference in the primary end point favoring the experimental treatment. The median ratio of observed-to-expected hazard ratios among trials that observed a statistically significant effect on the primary end point was 1.07 (range: 0.33-1.28) versus 1.32 (range: 0.86-2.02) for trials that did not, compared with 1.34 and 1.86, respectively, for National Cancer Institute (NCI) trials published between 1955 and 2006. An effect size at least as large as the one projected in the protocol trials was observed in 9.8% of trials, compared with 17% of NCI trials published from 1955 to 2006. Most trials (64.6%) provided no rationale to support the magnitude of the proposed treatment effect in the protocol.

Conclusions

Despite a reduction in 'optimism bias' compared with previous eras, most contemporary NCTN phase III trials failed to establish statistically significant benefits of new cancer therapies. Better rationalization of proposed effect sizes in research protocols is needed.

Many UC-authored scholarly publications are freely available on this site because of the UC's open access policies. Let us know how this access is important for you.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View