Proceedings of the Annual Meeting of the Cognitive Science Society

We examine the heterogeneity of text-based behavioral interventions in a series of 5 preregistered studies across one in-person and 10 online panels, with over 11000 respondents in total. We observe large heterogeneity across settings and paradigms. Model the heterogeneity we introduce a framework that measures typically omitted moderators: Fluid Intelligence, Attentiveness, Crystallized Intelligence, and Experience. Variation in these factors are associated with different effect sizes and explain variations across samples. Moderators are associated with effect sizes through two paths, with the intensity of the manipulation and with the effect of the manipulation directly. Our results motivate observing these moderators and provide a theoretical and empirical framework for understanding and predicting varying effect sizes in the social sciences.