- Main
Beyond Publication Bias: Characterizing and Understanding Missing Results in Economics
- Hoces de la Guardia, Fernando;
- Miguel, Edward;
- Pathan, Gufran;
- Shaji, Akash;
- Rocha, Viviane Helena Silva da;
- Sørensen, Erik Ø;
- Tungodden, Bertil
Published Web Location
https://doi.org/10.1257/rct.13755-2.0Abstract
What scientists choose not to publish can be as important as what they do publish. We measure missing research results at the hypothesis level and use this approach to study under-reporting in economics. Across 5,057 pre-registered hypotheses from 317 studies registered on the American Economic Association RCT Registry between 2015 and 2017, only 42% have publicly available results eight to nine years after pre-registration. Among the available results, 64% are null, and we find little evidence that null results are disproportionately missing. In an RCT, direct engagement with authors increases the share of hypotheses with available results by 6.2 percentage points, while raising awareness alone has no significant effect and additional research assistance provides no further improvement. Direct engagement also increases explanations for unavailable results, reducing the share of hypotheses that remain unaccounted for by 16.4 percentage points. The findings document extensive under-reporting in economics that does not appear to be driven primarily by the suppression of null results, and suggest that reporting pre-registered results may need to become a standard part of the research process rather than something recovered years later.