Critical considerations on certain topics in psychological science
Abstract
The field of psychology has entered a period of reform in which formative results are being doubted and traditional practices are being questioned. My research is focused on determining which findings and which methods we can trust.
In Chapter 1, I give a general introduction to Bayesian inference. Bayesian methods are becoming increasingly popular in psychology, as researchers recognize that traditional methods such as p-values cannot actually provide an answer to many of their research questions. Using seven worked examples, I illustrate the principles of Bayes’s rule, parameter estimation, and model comparison.
In Chapter 2, I revisit the results of the Reproducibility Project: Psychology by the Open Science Collaboration. I compute Bayes factors to quantify the amount of evidence provided by both the original and replication studies. I take into account the likely scenario that publication bias has distorted the originally published results by using a formal model ofthe biasing process. I conclude that the apparent failure of the Reproducibility Project to replicate many target effects can be adequately explained by overestimation of effect sizes due to small sample sizes and publication bias in the literature.
In Chapter 3, I extend our model of publication bias to allow for hierarchical modeling and meta-analysis. With this model it is possible to jointly model yoked sets of biased and unbiased studies and obtain more realistic meta-analytic results. I compare this model’s bias correction performance to other standard bias correction models using real (as opposed tosimulated) data.
In Chapter 4, I critically examine the HDI+ROPE hypothesis testing procedure. I demonstrate that this procedure is not internally consistent and not robust, because the ultimate inferential decision depends on statistically arbitrary and scientifically irrelevant properties of the statistical model. I identify the source of this issue, and conclude with recommendations for alternative Bayesian testing procedures that do not exhibit this pathology.