Land surface models range in complexity of terrestrial evapotranspiration, yet it is unknown how model complexity translates to accuracy of modeled evapotranspiration estimates. Here, we use the International Land Model Benchmarking system to assess ET estimates from three models of varying complexity driven by the same forcing datasets: an earth system model, a terrestrial biosphere model, and a stand-alone ET model. The performance assessment includes both temporal and spatial evaluation, and different plant functional types across China. Our results indicate that the most complex model, an earth system model, performed best against the benchmarking datasets and metrics. Terrestrial biosphere model performed best in simulating inter-annual variability of ET, while earth system model performed best in simulating the seasonal cycle. The more complex models (earth system model and terrestrial biosphere model) perform better in forest, shrub and crop ecosystems, while the simpler model (stand-alone ET model) perform better in grass ecosystems. Our study demonstrates the impact of model complexity on ET estimates and highlights directions for future ET model improvements.

This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had “in the pipeline” as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed.

We present the data from a crowdsourced project seeking to replicate findings in independent laboratories before (rather than after) they are published. In this Pre-Publication Independent Replication (PPIR) initiative, 25 research groups attempted to replicate 10 moral judgment effects from a single laboratory's research pipeline of unpublished findings. The 10 effects were investigated using online/lab surveys containing psychological manipulations (vignettes) followed by questionnaires. Results revealed a mix of reliable, unreliable, and culturally moderated findings. Unlike any previous replication project, this dataset includes the data from not only the replications but also from the original studies, creating a unique corpus that researchers can use to better understand reproducibility and irreproducibility in science.