UC Santa Barbara

Educational researchers are often tasked with postulating and measuring unknown processes that account for observed outcomes. Logically, this requires inductive reasoning about which constructs are relevant to the situation we seek to understand, prior to attempts at measurement. Exploratory mixed methods designs, along with a few classic designs from psychometrics, represent the canonical approaches to addressing this research problem. However, I argue that these canonical approaches require rethinking in order to fully embrace both the logic of discovery and the philosophy of measurement. Four normative strands are presented to guide this rethinking: an emphasis on the logic of discovery alongside the customary logic of justification, a move towards radical transparency, an invitation to philosophical exploration, and an appeal to integrate qualitative analysis at every major step of the measurement process. The result of following these normative strands would be a discovery-based workflow for educational measurement. To illustrate such a workflow, I take as an example data from a program evaluation of an intervention for underrepresented minority students in a university STEM program.