Proceedings of the Annual Meeting of the Cognitive Science Society

Recently Milkman et al. (2021) reported that heterogeneity of outcomes is extremely difficult to predict in advance and that behavioral experts, despite their educational background or their extensive applied research experience, might not be better in their predictions than non-experts. Such findings are rather concerning and deserve more attention. In the current project we present three studies aimed at assessing the quality of human predictions in respect to outcome heterogeneity of behavioral interventions. We borrow methods from anthropology and cognitive psychology to asses both consensus and accuracy of such predictions as well as to evaluate the potential role of behavioral expertise. Our results are more optimistic than previous findings and reveal that experts’ predictions are significantly better than a chance level. These findings have implications for applied behavioral research and further advance our theoretical understanding of expertise.