Skip to main content
eScholarship
Open Access Publications from the University of California

Reproducible biomedical benchmarking in the cloud: lessons from crowd-sourced data challenges.

  • Author(s): Ellrott, Kyle
  • Buchanan, Alex
  • Creason, Allison
  • Mason, Michael
  • Schaffter, Thomas
  • Hoff, Bruce
  • Eddy, James
  • Chilton, John M
  • Yu, Thomas
  • Stuart, Joshua M
  • Saez-Rodriguez, Julio
  • Stolovitzky, Gustavo
  • Boutros, Paul C
  • Guinney, Justin
  • et al.

Published Web Location

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6737594/
No data is associated with this publication.
Abstract

Challenges are achieving broad acceptance for addressing many biomedical questions and enabling tool assessment. But ensuring that the methods evaluated are reproducible and reusable is complicated by the diversity of software architectures, input and output file formats, and computing environments. To mitigate these problems, some challenges have leveraged new virtualization and compute methods, requiring participants to submit cloud-ready software packages. We review recent data challenges with innovative approaches to model reproducibility and data sharing, and outline key lessons for improving quantitative biomedical data analysis through crowd-sourced benchmarking challenges.

Many UC-authored scholarly publications are freely available on this site because of the UC's open access policies. Let us know how this access is important for you.

Item not freely available? Link broken?
Report a problem accessing this item