Center for Effective Global Action

This mixed methods study investigates why fewer than one in five impact evaluations integrates a value-for-money analysis of the development intervention being evaluated. This study distills four main insights from combined analysis of 33 semi-structured and unstructured interviews, surveys of 497 policy makers and 16 journal editors, and portfolio analyses of World Bank and worldwide impact evaluations. The study finds that low levels of training in cost data collection and analysis methods, together with a lack of standardization of the value-for-money assumptions (e.g., time horizons, discount rates, and economic or financial cost accounting) limit value-for-money integration into impact evaluations. Further eroding researchers' incentives, demand for cost evidence from the journals that publish impact evaluations is mixed. Ill-defined standards of rigor undermine editors' capacity to evaluate the quality of value-for-money analysis when it is integrated with impact evaluation evidence. Institutional funders of impact evaluations do not consistently demand that cost analysis be integrated into their funded evaluations. This study finds no evidence in support of the myth that policymakers do not demand cost evidence. Rather, it finds that researchers have few ways of knowing what kind of analysis policymakers need and when they need it. Improving the stock of impact evaluators who are cross trained in value-for-money methods, establishing standards in what constitutes rigor in costing, resolving methodological issues, and improving linkages between policymakers and researchers would lead to greater integration of value-for-money methods in impact evaluations.